-- Logs begin at Fri 2022-05-27 14:22:51 UTC, end at Fri 2022-05-27 15:34:05 UTC. -- May 27 14:59:34.450567 dv-cinder-842542-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 27 14:59:34.459877 dv-cinder-842542-2-iscsi devstack@keystone.service[80387]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 27 14:59:34.462158 dv-cinder-842542-2-iscsi devstack@keystone.service[80387]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri May 27 14:59:34 2022] *** May 27 14:59:34.462158 dv-cinder-842542-2-iscsi devstack@keystone.service[80387]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 27 14:59:34.462158 dv-cinder-842542-2-iscsi devstack@keystone.service[80387]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 27 14:59:34.462158 dv-cinder-842542-2-iscsi devstack@keystone.service[80387]: nodename: dv-cinder-842542-2-iscsi May 27 14:59:34.462158 dv-cinder-842542-2-iscsi devstack@keystone.service[80387]: machine: x86_64 May 27 14:59:34.462158 dv-cinder-842542-2-iscsi devstack@keystone.service[80387]: clock source: unix May 27 14:59:34.462158 dv-cinder-842542-2-iscsi devstack@keystone.service[80387]: pcre jit disabled May 27 14:59:34.462158 dv-cinder-842542-2-iscsi devstack@keystone.service[80387]: detected number of CPU cores: 4 May 27 14:59:34.462158 dv-cinder-842542-2-iscsi devstack@keystone.service[80387]: current working directory: / May 27 14:59:34.462158 dv-cinder-842542-2-iscsi devstack@keystone.service[80387]: detected binary path: /usr/bin/uwsgi-core May 27 14:59:34.462826 dv-cinder-842542-2-iscsi devstack@keystone.service[80387]: your processes number limit is 47680 May 27 14:59:34.462826 dv-cinder-842542-2-iscsi devstack@keystone.service[80387]: your memory page size is 4096 bytes May 27 14:59:34.462826 dv-cinder-842542-2-iscsi devstack@keystone.service[80387]: detected max file descriptor number: 2048 May 27 14:59:34.462826 dv-cinder-842542-2-iscsi devstack@keystone.service[80387]: lock engine: pthread robust mutexes May 27 14:59:34.462826 dv-cinder-842542-2-iscsi devstack@keystone.service[80387]: thunder lock: enabled May 27 14:59:34.462826 dv-cinder-842542-2-iscsi devstack@keystone.service[80387]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 27 14:59:34.462826 dv-cinder-842542-2-iscsi devstack@keystone.service[80387]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 27 14:59:34.507282 dv-cinder-842542-2-iscsi devstack@keystone.service[80387]: Python main interpreter initialized at 0x555b0860c9c0 May 27 14:59:34.507282 dv-cinder-842542-2-iscsi devstack@keystone.service[80387]: python threads support enabled May 27 14:59:34.507282 dv-cinder-842542-2-iscsi devstack@keystone.service[80387]: your server socket listen backlog is limited to 100 connections May 27 14:59:34.507282 dv-cinder-842542-2-iscsi devstack@keystone.service[80387]: your mercy for graceful operations on workers is 90 seconds May 27 14:59:34.507614 dv-cinder-842542-2-iscsi devstack@keystone.service[80387]: mapped 403077 bytes (393 KB) for 2 cores May 27 14:59:34.507691 dv-cinder-842542-2-iscsi devstack@keystone.service[80387]: *** Operational MODE: preforking *** May 27 14:59:34.507951 dv-cinder-842542-2-iscsi devstack@keystone.service[80387]: *** uWSGI is running in multiple interpreter mode *** May 27 14:59:34.507951 dv-cinder-842542-2-iscsi devstack@keystone.service[80387]: spawned uWSGI master process (pid: 80387) May 27 14:59:34.508143 dv-cinder-842542-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 27 14:59:34.510887 dv-cinder-842542-2-iscsi devstack@keystone.service[80387]: spawned uWSGI worker 1 (pid: 80394, cores: 1) May 27 14:59:34.510887 dv-cinder-842542-2-iscsi devstack@keystone.service[80387]: spawned uWSGI worker 2 (pid: 80395, cores: 1) May 27 14:59:34.510887 dv-cinder-842542-2-iscsi devstack@keystone.service[80387]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 27 14:59:40.211372 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 27 14:59:40.211372 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: from cryptography.utils import int_from_bytes May 27 14:59:40.212368 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 27 14:59:40.212368 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: from cryptography.utils import int_from_bytes May 27 14:59:40.212662 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 27 14:59:40.212662 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: from cryptography.utils import int_from_bytes May 27 14:59:40.213117 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 27 14:59:40.213117 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: from cryptography.utils import int_from_bytes May 27 14:59:40.261350 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 27 14:59:40.261539 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 27 14:59:40.261623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 27 14:59:40.261731 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] command line args: [] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 27 14:59:40.261818 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 27 14:59:40.261927 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] command line args: [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 27 14:59:40.262004 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 27 14:59:40.262162 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ================================================================================ {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 27 14:59:40.262248 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 27 14:59:40.262368 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ================================================================================ {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 27 14:59:40.262445 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] admin_token = **** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.262625 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] admin_token = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.262710 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] config_dir = [] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.262816 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.262913 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] config_dir = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.263026 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] config_source = [] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.263112 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.263196 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] debug = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.263276 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] config_source = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.263388 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.263641 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] debug = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.263641 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.263794 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.263856 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.263920 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.264064 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.264183 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.264288 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.264385 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.264480 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.264609 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] list_limit = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.264720 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] list_limit = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.264826 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] log_config_append = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.264923 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] log_config_append = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.265017 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.265119 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.265220 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] log_dir = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.265322 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] log_dir = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.265430 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] log_file = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.265542 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] log_file = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.265645 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.265725 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.265842 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.265918 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.266043 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.266122 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.266238 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.266322 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.266396 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.266511 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.266600 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.266681 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.266754 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.266882 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.266995 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.267132 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.267213 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.267371 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.267449 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.268132 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.268287 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.268381 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.268526 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.268615 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.268774 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.268863 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.269020 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.269151 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.269295 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.269374 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.269776 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.50/identity {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.269854 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.50/identity {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.269981 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] publish_errors = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.270064 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] publish_errors = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.270172 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.270258 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.270399 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.270482 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.270576 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.270658 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.270818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.270981 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.271066 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.271184 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.271294 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] standard_threads = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.271380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] standard_threads = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.271521 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.271605 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.271761 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.271849 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.271928 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.272005 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.272085 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] use_journal = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.272163 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] use_journal = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.272256 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] use_json = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.272340 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] use_json = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.272420 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] use_stderr = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.272491 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] use_stderr = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.272570 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] use_syslog = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.272647 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] use_syslog = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.272800 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.272879 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 27 14:59:40.273054 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.273133 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.273291 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.273375 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.273548 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.273632 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.273783 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.273869 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.274029 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.274106 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.274264 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.274342 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.274510 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.274663 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.274738 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.274809 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.275002 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.275083 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.275247 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.275337 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.275499 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.275582 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.275729 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.275812 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.275975 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.276060 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.276208 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.276290 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.276452 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.276529 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.276695 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.276781 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.276930 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.277014 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.277181 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.277259 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.277436 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.277521 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.277678 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.277755 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.277912 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.277989 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.278161 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.278246 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.278418 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.278495 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.278666 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.278751 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.278938 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.279031 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.279168 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.279245 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.279397 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.279475 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.279624 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.279712 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.279857 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.279941 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.280102 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.280185 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.280347 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.280500 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.280575 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.280645 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.280790 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.280870 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.281036 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.281113 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.281276 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.281353 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.281510 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.281589 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.281757 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.281837 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.281987 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.282065 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.282221 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.282298 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.282449 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.282528 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.282678 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.282756 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.282928 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.283009 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.283745 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.283833 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.283990 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.284067 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.284264 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.284362 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.284516 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.284599 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.284750 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.284859 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.284991 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.285104 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.285211 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.285315 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.285451 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.285555 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.285690 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.285797 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.285943 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.286032 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.286181 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.286265 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.286408 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.286535 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.286623 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.286695 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.286890 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.286989 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.287104 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.287183 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.287339 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.287414 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.287572 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.287649 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.287813 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.287909 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.288049 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.288148 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.288280 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.288371 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.288508 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.288594 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.288762 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.288841 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.288996 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.289077 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.289218 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.289327 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.289452 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.289549 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.289684 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.289782 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.289948 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.290057 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.290184 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.290281 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.290432 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.290518 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.290671 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.290743 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.290933 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.291034 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.291166 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.291245 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.291400 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.291478 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.291647 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.291724 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.291881 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.291959 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.292115 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.292193 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.292358 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.292436 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.292586 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.292660 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.292817 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.292895 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.293050 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.293136 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.293284 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.293362 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.293516 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.293594 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.293777 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.293870 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.294014 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.294096 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.294252 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.294329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.294487 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.294573 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.294727 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.294805 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.294987 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.295067 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.295221 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.295310 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.295455 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.295550 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.295684 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.295763 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.295913 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.295992 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.296142 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.296227 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.296372 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.296450 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.296599 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.296677 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.296838 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.296923 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.297069 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.297146 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.297310 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.297395 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.297549 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.297635 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.297800 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.297876 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.298056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.298134 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.298294 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.298380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.298524 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] auth.external = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.298601 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] auth.external = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.298782 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.298883 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.299113 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.299355 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.299446 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.299528 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.299602 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] auth.password = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.299675 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] auth.password = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.299793 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] auth.token = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.299906 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] auth.token = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.300035 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.300144 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.300273 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.300378 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.300522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.300621 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.300757 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.300837 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.301007 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.301085 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.301257 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.301343 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.301491 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.301581 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.301731 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] credential.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.301843 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] credential.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.301979 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.302068 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.302221 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.302310 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.302466 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.302562 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.302697 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.302792 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.302963 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.303053 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.303198 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.303294 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.303452 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.303581 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.303714 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.303847 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.303977 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.304104 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.304241 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.304369 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.304503 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.304631 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.304757 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.304882 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.305011 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.305137 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.305272 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.305399 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.305530 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.305657 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.305783 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] federation.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.305909 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.306055 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.306188 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.306315 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.306449 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.306576 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.306702 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.306927 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.307055 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.307182 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.307310 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.307437 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.307564 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.307698 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.307843 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.307972 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.308101 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.308234 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.308366 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.308497 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.308624 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.308750 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.308876 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.309002 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.309135 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.309271 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.309398 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.309525 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.309666 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.309794 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.309928 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.310063 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.310194 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.310322 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.310448 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.310574 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.310708 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.310835 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.311011 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.311140 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.311262 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.311403 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.311541 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.311674 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.311795 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.311922 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.312049 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.312193 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.312329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.312408 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.312481 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.312553 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.312625 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.312697 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.312776 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.312850 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.312850 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.312980 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.312980 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.313107 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.313107 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.313245 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.313364 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.313441 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.313513 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.313584 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.313664 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.313738 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.313809 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.313881 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.313952 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.314023 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.314102 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.314175 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.314246 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.314941 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.315173 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.315256 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.315343 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.315425 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.315550 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.315656 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.315737 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.315896 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.315982 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.316141 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.316220 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.316378 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.316456 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.316604 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.316690 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.316868 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.316946 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.317102 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.317181 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.317343 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.317430 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.317574 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.317654 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.317810 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.317888 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.318051 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.318170 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.318290 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.318380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.318533 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.318620 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.318773 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.318876 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.319025 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.319104 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.319254 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.319339 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.319500 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.319591 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.319738 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.319817 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.319964 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.320042 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.320198 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.320303 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.320417 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.320531 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.320646 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.320725 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.320883 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.320970 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.321113 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.321235 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.321354 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.321473 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.321615 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.321703 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.321847 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.321926 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.322095 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.322173 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.322344 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.322424 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.322591 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.322670 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.322822 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.322964 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.323076 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.323157 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.323309 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.323387 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.323539 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.323618 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.323777 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.323857 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.324009 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.324088 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.324250 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.324327 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.324496 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.324575 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.324738 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.324819 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.324968 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.325046 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.325211 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.325292 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.325441 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.325519 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.325672 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.325775 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.325901 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.325980 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.326124 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.326204 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.326365 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.326450 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.326610 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.326688 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.326843 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.326940 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.327113 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.327192 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.327345 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.327426 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.327586 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.327663 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.327820 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.327897 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.328064 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.328143 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.328297 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.328375 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.328537 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.328614 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.328802 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.328893 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.328999 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.329078 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.329232 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.329310 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.329458 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.329532 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] resource.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.329693 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.329772 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.329934 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.330052 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.330750 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.330904 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.331042 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.331172 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.331342 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.331489 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.331624 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.331751 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.331877 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.332003 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.332128 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.332250 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.332385 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.332513 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.332638 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] role.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.332764 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.332904 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] role.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.333030 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.333164 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.333293 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] role.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.333426 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.333554 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] role.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.333680 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.333808 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.333942 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.334069 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.334196 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.334323 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.334450 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.334587 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.334721 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.334895 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.335021 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.335148 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.335275 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.335413 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.335539 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.335666 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.335793 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.335919 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.336045 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.336184 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.336320 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.336449 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.336576 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.336703 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.336829 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.336963 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.337364 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.337487 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.337614 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.337741 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.337868 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.338003 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.338130 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.338266 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.338394 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.338394 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.338634 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.338767 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.338909 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.339031 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.339157 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.339284 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.339419 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.339554 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.339683 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.339809 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.339935 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.340073 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.340200 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.340335 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.340417 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.340583 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.340705 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.340827 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.340950 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.341081 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.341206 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.341329 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.341451 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.341574 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.341714 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.341838 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] token.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.341960 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.342084 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.342207 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.342330 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.342459 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.342583 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.342707 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.342830 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.342984 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.343112 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.343248 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.343376 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.343512 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.343638 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.343766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.343892 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.344026 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.344153 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.344279 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.344405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.344530 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.344656 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.344790 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.344918 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.345043 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.345179 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.345305 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.345432 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.345566 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.345711 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.345838 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.345964 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.346090 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.346217 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.346352 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.346479 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.connection = **** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.346613 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.346748 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.346889 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.347023 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.347147 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.347147 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.connection = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.347393 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.347529 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.347653 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.347776 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.347899 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.348023 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.348145 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.348276 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.348400 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.348523 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.348645 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.348778 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.348901 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.349032 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.349155 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.349279 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.349401 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.349523 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.349645 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.349775 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.349899 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.350022 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.350144 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.350268 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.350396 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.350604 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.350730 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.350866 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.350991 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.351116 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.351238 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.351238 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80394) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 27 14:59:40.351482 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.351482 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.351482 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 27 14:59:40.351482 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 27 14:59:40.379406 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.379629 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.379841 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.381174 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.381411 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.381642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.390068 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.390186 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.418725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.418915 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.419082 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.419320 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.471958 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.473078 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.522641 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.522889 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.523537 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.523774 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.588264 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.588483 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.588643 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.588731 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.588880 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.588966 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.589107 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.589190 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.589317 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.589409 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.589538 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.589757 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.613622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.613854 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.613963 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.614076 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.614160 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.614302 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.614385 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.614521 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.614605 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.614732 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.614814 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.614967 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.615052 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.615215 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.626183 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.626521 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.638919 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.639031 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.639171 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.639257 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.639333 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.639407 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.639544 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.639636 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.639762 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.639845 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.639977 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.640059 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.640189 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80394) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.640279 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 27 14:59:40.657570 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.657777 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 27 14:59:40.657923 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.658613 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.658719 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 27 14:59:40.658902 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 27 14:59:40.659130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.659221 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.659296 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 27 14:59:40.659414 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.659555 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.659663 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.659816 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.659896 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.660088 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.660328 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.671365 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.671602 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.671872 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.672153 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.672436 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.672707 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.672968 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.673239 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.680937 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:59:40.681996 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:59:40.684261 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:59:40.684544 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.684825 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.685106 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.685275 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:59:40.685386 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.685552 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.685635 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.685827 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.686011 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.686091 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.686290 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.686372 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.686565 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.686715 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.686841 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.686999 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.687138 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.687267 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.687535 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.687805 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.688074 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.714433 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.715097 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.716223 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:59:40.716505 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.716792 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:59:40.717067 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.721855 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:59:40.722130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.722382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:59:40.722465 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.722657 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.722968 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.729733 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:59:40.730017 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.730243 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:59:40.730520 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.734499 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:59:40.734924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.735008 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:59:40.735211 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.735366 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.735492 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.735643 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.735769 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.735918 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.736045 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.736191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.736831 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.737061 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.737169 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.737329 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.737597 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.756345 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.756623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.757009 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.757283 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.761275 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:59:40.761909 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:59:40.764911 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.765208 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.765629 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.765957 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.770645 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:59:40.771010 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:59:40.774278 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:59:40.774578 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:59:40.778065 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.778336 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.778426 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.778627 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.778710 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.778936 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.779103 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.779218 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.779301 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.779500 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.779583 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.779781 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.798493 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:59:40.798637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:59:40.799345 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.799521 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.799647 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.799791 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.799928 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.800143 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.808146 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.808406 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.809871 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.810140 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.810314 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.810593 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.813210 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:59:40.813499 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.813759 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:59:40.813977 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.814218 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.815025 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.815305 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.815529 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.815529 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.815936 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.828853 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:59:40.829119 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.829409 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.829691 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.830300 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.830449 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:59:40.830600 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.830786 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.830907 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.830987 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.830987 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.831172 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.831442 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.831712 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.849209 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:59:40.849940 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:59:40.852502 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:59:40.853209 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:59:40.855858 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.856589 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.857459 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.858192 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.859115 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:59:40.859409 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.859693 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.859814 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:59:40.860097 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.860382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.867753 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.868309 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.869360 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:59:40.869904 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:59:40.872705 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.873006 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.873163 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.873293 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.873446 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.873574 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.873725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.874006 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.883654 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:59:40.883753 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:59:40.883945 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.884041 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.884320 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.884404 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.892123 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80394) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:59:40.892214 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 27 14:59:40.892399 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.892519 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.892795 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.892925 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.893065 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.893148 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.893332 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.893420 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.893599 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.893682 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.893876 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.893967 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.894156 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.894237 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.894420 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.894554 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.894686 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.894768 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.894980 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.895065 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.895239 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.895326 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.895506 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.895587 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.895767 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80394) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.895847 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 27 14:59:40.935653 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x555b0860c9c0 pid: 80394 (default app) May 27 14:59:40.937165 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x555b0860c9c0 pid: 80395 (default app) May 27 14:59:40.938781 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86eee2b-f0f2-4d3f-8975-1186f6682cfa None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:40.939106 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86eee2b-f0f2-4d3f-8975-1186f6682cfa None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:40.940046 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86eee2b-f0f2-4d3f-8975-1186f6682cfa None None] PATH_INFO: `/v3` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:40.944729 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1/1] 10.222.0.50 () {50 vars in 791 bytes} [Fri May 27 14:59:40 2022] GET /identity/v3/ => generated 255 bytes in 9 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:48.931407 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7cd831-b23a-4fce-82ec-1fc5773ee863 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:48.931969 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7cd831-b23a-4fce-82ec-1fc5773ee863 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:48.931969 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7cd831-b23a-4fce-82ec-1fc5773ee863 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:48.936349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1/2] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 14:59:48 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 14:59:48.942101 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f111ec-dc0c-4bbe-9a63-18ffab0ab951 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:48.942488 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f111ec-dc0c-4bbe-9a63-18ffab0ab951 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:48.942648 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f111ec-dc0c-4bbe-9a63-18ffab0ab951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:49.010258 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: INFO dbcounter [None req-30f111ec-dc0c-4bbe-9a63-18ffab0ab951 None None] Registered counter for database keystone May 27 14:59:49.031485 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG oslo_db.sqlalchemy.engines [None req-30f111ec-dc0c-4bbe-9a63-18ffab0ab951 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80394) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 27 14:59:49.037795 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writer thread running {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 27 14:59:49.212420 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG passlib.handlers.bcrypt [None req-30f111ec-dc0c-4bbe-9a63-18ffab0ab951 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80394) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 27 14:59:49.212784 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG passlib.handlers.bcrypt [None req-30f111ec-dc0c-4bbe-9a63-18ffab0ab951 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80394) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 27 14:59:49.268188 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-30f111ec-dc0c-4bbe-9a63-18ffab0ab951 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:49.269781 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-30f111ec-dc0c-4bbe-9a63-18ffab0ab951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:49.396893 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 2/3] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 14:59:48 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 456 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 27 14:59:49.574172 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7cd831-b23a-4fce-82ec-1fc5773ee863 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:49.642754 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: INFO dbcounter [None req-8b7cd831-b23a-4fce-82ec-1fc5773ee863 None None] Registered counter for database keystone May 27 14:59:49.662348 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG oslo_db.sqlalchemy.engines [None req-8b7cd831-b23a-4fce-82ec-1fc5773ee863 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80395) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 27 14:59:49.668460 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writer thread running {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 27 14:59:49.786556 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7cd831-b23a-4fce-82ec-1fc5773ee863 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:49.790733 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e513c2f2-9147-42a2-94dd-9451d19f22fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:49.791411 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e513c2f2-9147-42a2-94dd-9451d19f22fa admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:49.791691 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e513c2f2-9147-42a2-94dd-9451d19f22fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:49.791965 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e513c2f2-9147-42a2-94dd-9451d19f22fa admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:49.800007 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e513c2f2-9147-42a2-94dd-9451d19f22fa admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:49.850824 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e513c2f2-9147-42a2-94dd-9451d19f22fa admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 27 14:59:49.851131 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e513c2f2-9147-42a2-94dd-9451d19f22fa admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:49.860462 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 2/4] 10.222.0.50 () {58 vars in 1324 bytes} [Fri May 27 14:59:49 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 14:59:49.866562 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f111ec-dc0c-4bbe-9a63-18ffab0ab951 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:49.880963 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f111ec-dc0c-4bbe-9a63-18ffab0ab951 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:49.885474 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6946716e-b64d-4dcd-98dd-361ad793ed42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:49.886602 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946716e-b64d-4dcd-98dd-361ad793ed42 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:49.886923 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946716e-b64d-4dcd-98dd-361ad793ed42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:49.887204 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946716e-b64d-4dcd-98dd-361ad793ed42 admin admin] PATH_INFO: `/v3/projects/6fdd45226b7e4d17bd9b93b1a5b70196` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:49.888768 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6946716e-b64d-4dcd-98dd-361ad793ed42 admin admin] RBAC: Authorizing `identity:get_project(project_id=6fdd45226b7e4d17bd9b93b1a5b70196)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:49.941375 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: WARNING keystone.common.rbac_enforcer.enforcer [None req-6946716e-b64d-4dcd-98dd-361ad793ed42 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 27 14:59:49.941713 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6946716e-b64d-4dcd-98dd-361ad793ed42 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:49.946236 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 3/5] 10.222.0.50 () {56 vars in 1207 bytes} [Fri May 27 14:59:49 2022] GET /identity/v3/projects/6fdd45226b7e4d17bd9b93b1a5b70196 => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:52.214699 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee743489-5d39-46cd-a0af-198fb157b825 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:52.215218 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee743489-5d39-46cd-a0af-198fb157b825 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:52.215306 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee743489-5d39-46cd-a0af-198fb157b825 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:52.219386 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 3/6] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 14:59:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 14:59:52.225231 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ceb50-b80c-4fa8-ad16-11781947eb91 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:52.225642 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ceb50-b80c-4fa8-ad16-11781947eb91 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:52.225793 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ceb50-b80c-4fa8-ad16-11781947eb91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:52.244461 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-896ceb50-b80c-4fa8-ad16-11781947eb91 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:52.245959 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-896ceb50-b80c-4fa8-ad16-11781947eb91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:52.253843 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 4/7] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 14:59:52 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 27 14:59:52.275822 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee743489-5d39-46cd-a0af-198fb157b825 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:52.288546 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee743489-5d39-46cd-a0af-198fb157b825 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:52.292715 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495cf4a9-82e0-4e03-be51-93bc0bd2a666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:52.293338 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495cf4a9-82e0-4e03-be51-93bc0bd2a666 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:52.293623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495cf4a9-82e0-4e03-be51-93bc0bd2a666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:52.293900 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495cf4a9-82e0-4e03-be51-93bc0bd2a666 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:52.301421 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495cf4a9-82e0-4e03-be51-93bc0bd2a666 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:52.303271 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495cf4a9-82e0-4e03-be51-93bc0bd2a666 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:52.311794 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 4/8] 10.222.0.50 () {58 vars in 1324 bytes} [Fri May 27 14:59:52 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 14:59:52.317469 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ceb50-b80c-4fa8-ad16-11781947eb91 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:52.326332 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896ceb50-b80c-4fa8-ad16-11781947eb91 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:52.331420 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290f54cf-8f3b-4cf4-8bbb-7e4cc5f6175c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:52.332272 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f54cf-8f3b-4cf4-8bbb-7e4cc5f6175c admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:52.332555 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f54cf-8f3b-4cf4-8bbb-7e4cc5f6175c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:52.332832 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290f54cf-8f3b-4cf4-8bbb-7e4cc5f6175c admin admin] PATH_INFO: `/v3/users/9c29995c0b564cd7879c58c75dc099cb` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:52.334201 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290f54cf-8f3b-4cf4-8bbb-7e4cc5f6175c admin admin] RBAC: Authorizing `identity:get_user(user_id=9c29995c0b564cd7879c58c75dc099cb)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:52.336052 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290f54cf-8f3b-4cf4-8bbb-7e4cc5f6175c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:52.337875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 5/9] 10.222.0.50 () {56 vars in 1198 bytes} [Fri May 27 14:59:52 2022] GET /identity/v3/users/9c29995c0b564cd7879c58c75dc099cb => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:54.595273 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecbc406-799a-4dcc-bcd4-5a9c88cef680 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:54.595785 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecbc406-799a-4dcc-bcd4-5a9c88cef680 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:54.595887 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecbc406-799a-4dcc-bcd4-5a9c88cef680 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:54.597162 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 5/10] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 14:59:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:59:54.602405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc1936-17f7-4547-bc27-9feb49dc044b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:54.602841 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc1936-17f7-4547-bc27-9feb49dc044b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:54.603007 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc1936-17f7-4547-bc27-9feb49dc044b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:54.619375 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-55dc1936-17f7-4547-bc27-9feb49dc044b None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:54.620542 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-55dc1936-17f7-4547-bc27-9feb49dc044b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:54.650885 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd834cb0-20a0-4218-9138-bb5927119bf0 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:54.651178 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd834cb0-20a0-4218-9138-bb5927119bf0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:54.651456 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd834cb0-20a0-4218-9138-bb5927119bf0 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:54.652795 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 6/11] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 14:59:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 14:59:54.657724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486dad1-8529-43de-8ea4-d2ab65e05807 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:54.658048 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486dad1-8529-43de-8ea4-d2ab65e05807 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:54.658325 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486dad1-8529-43de-8ea4-d2ab65e05807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:54.697626 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG passlib.handlers.bcrypt [None req-9486dad1-8529-43de-8ea4-d2ab65e05807 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80395) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 27 14:59:54.698056 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG passlib.handlers.bcrypt [None req-9486dad1-8529-43de-8ea4-d2ab65e05807 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80395) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 27 14:59:54.700779 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 6/12] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 14:59:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 14:59:54.724609 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc1936-17f7-4547-bc27-9feb49dc044b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:54.744949 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9486dad1-8529-43de-8ea4-d2ab65e05807 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:54.746513 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9486dad1-8529-43de-8ea4-d2ab65e05807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:54.747665 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:54.827986 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc1936-17f7-4547-bc27-9feb49dc044b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:54.847352 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 7/13] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 14:59:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 14:59:54.863161 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d69d8-54d2-4145-bebe-51ebcac20c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:54.863954 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d69d8-54d2-4145-bebe-51ebcac20c6d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:54.864240 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d69d8-54d2-4145-bebe-51ebcac20c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:54.864518 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d69d8-54d2-4145-bebe-51ebcac20c6d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:54.868945 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9486dad1-8529-43de-8ea4-d2ab65e05807 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:54.875984 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3d69d8-54d2-4145-bebe-51ebcac20c6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:59:54.887656 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-8f3d69d8-54d2-4145-bebe-51ebcac20c6d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 14:59:54.887656 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:54.887656 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:54.887656 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:54.887656 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:54.887656 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:54.887656 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:54.887656 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:54.887656 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:54.887656 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:54.887656 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:54.887656 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:54.887656 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:59:54.887656 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:59:54.887656 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:59:54.887656 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:59:54.888837 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:59:54.888837 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:59:54.888837 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:59:54.888837 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:59:54.888837 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:54.888837 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:54.888837 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:54.888837 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:54.888837 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:54.888837 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:54.888837 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:54.888837 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 14:59:54.888837 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:54.888837 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:54.888837 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:54.888837 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:54.889683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:54.889683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:54.889683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:54.889683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:54.889683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:59:54.889683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:59:54.889683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:59:54.889683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:59:54.889683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:59:54.889683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:59:54.889683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 14:59:54.889683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 14:59:54.889683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 7/14] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 14:59:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 14:59:54.895289 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d69d8-54d2-4145-bebe-51ebcac20c6d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:55.080242 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9486dad1-8529-43de-8ea4-d2ab65e05807 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:55.082548 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:55.091148 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:55.109555 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d69d8-54d2-4145-bebe-51ebcac20c6d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:55.118872 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3b9ed0-d3f7-4fb1-81f8-dc5e19ef2836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:55.119794 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3b9ed0-d3f7-4fb1-81f8-dc5e19ef2836 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:55.120082 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3b9ed0-d3f7-4fb1-81f8-dc5e19ef2836 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:55.120357 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3b9ed0-d3f7-4fb1-81f8-dc5e19ef2836 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:55.143375 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce92ed70-b111-41d5-876c-5c568e03cdfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:55.144030 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92ed70-b111-41d5-876c-5c568e03cdfc None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:55.144314 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92ed70-b111-41d5-876c-5c568e03cdfc None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:55.144607 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce92ed70-b111-41d5-876c-5c568e03cdfc None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:55.145349 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce92ed70-b111-41d5-876c-5c568e03cdfc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:55.145657 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce92ed70-b111-41d5-876c-5c568e03cdfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:55.147118 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce92ed70-b111-41d5-876c-5c568e03cdfc None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:55.162120 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 8/15] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 14:59:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:59:55.167554 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce92ed70-b111-41d5-876c-5c568e03cdfc None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:55.188788 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3b9ed0-d3f7-4fb1-81f8-dc5e19ef2836 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:55.191188 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3b9ed0-d3f7-4fb1-81f8-dc5e19ef2836 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:55.266423 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce92ed70-b111-41d5-876c-5c568e03cdfc None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:55.288503 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 8/16] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 14:59:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 14:59:55.294019 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3b9ed0-d3f7-4fb1-81f8-dc5e19ef2836 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:55.296705 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:55.301375 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e7f04-e3b4-4c8f-ade3-da5c85376870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:55.302254 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e7f04-e3b4-4c8f-ade3-da5c85376870 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:55.302540 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e7f04-e3b4-4c8f-ade3-da5c85376870 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:55.302821 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e7f04-e3b4-4c8f-ade3-da5c85376870 None admin] PATH_INFO: `/v3/users/9c29995c0b564cd7879c58c75dc099cb` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:55.304066 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248e7f04-e3b4-4c8f-ade3-da5c85376870 None admin] RBAC: Authorizing `identity:get_user(user_id=9c29995c0b564cd7879c58c75dc099cb)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:55.305554 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248e7f04-e3b4-4c8f-ade3-da5c85376870 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:55.307197 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 9/17] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 14:59:55 2022] GET /identity/v3/users/9c29995c0b564cd7879c58c75dc099cb => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:59:55.312445 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e7f04-e3b4-4c8f-ade3-da5c85376870 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:55.327254 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:55.396723 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3b9ed0-d3f7-4fb1-81f8-dc5e19ef2836 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:55.412970 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e7f04-e3b4-4c8f-ade3-da5c85376870 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:55.430557 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0e2d3c-8b49-4205-9406-ccd87e2059ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:55.431385 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e2d3c-8b49-4205-9406-ccd87e2059ba None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:55.431676 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e2d3c-8b49-4205-9406-ccd87e2059ba None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:55.431954 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e2d3c-8b49-4205-9406-ccd87e2059ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:55.446323 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db78ebc7-bd73-41ac-a248-419a491fc755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:55.447158 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db78ebc7-bd73-41ac-a248-419a491fc755 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:55.447446 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db78ebc7-bd73-41ac-a248-419a491fc755 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:55.447725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db78ebc7-bd73-41ac-a248-419a491fc755 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:55.448909 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db78ebc7-bd73-41ac-a248-419a491fc755 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:55.450350 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db78ebc7-bd73-41ac-a248-419a491fc755 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:55.450585 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0e2d3c-8b49-4205-9406-ccd87e2059ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:55.451976 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 10/18] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 14:59:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:59:55.452104 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0e2d3c-8b49-4205-9406-ccd87e2059ba None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:55.457581 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db78ebc7-bd73-41ac-a248-419a491fc755 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:55.473575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-ec0e2d3c-8b49-4205-9406-ccd87e2059ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:55.473575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:55.473575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:55.473575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:55.473575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:55.473575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:55.473575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:55.473575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:55.473575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:55.473575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:55.473575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:55.473575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:55.473575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:59:55.473575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:59:55.473575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:59:55.474448 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:59:55.474448 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:55.474448 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:55.474448 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:55.474448 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:55.474448 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:55.474448 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:55.474448 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:55.474448 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 14:59:55.474448 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:55.474448 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:55.474448 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:55.474448 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:55.474448 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:55.474448 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:55.474448 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:55.474448 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:55.475520 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:59:55.475520 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:59:55.475520 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:55.475520 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 14:59:55.475520 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 9/19] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 14:59:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 14:59:55.480909 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0e2d3c-8b49-4205-9406-ccd87e2059ba None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:55.537978 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:55.539276 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:55.559359 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db78ebc7-bd73-41ac-a248-419a491fc755 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:55.583213 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0e2d3c-8b49-4205-9406-ccd87e2059ba None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:55.592759 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e67de-84af-4ea2-aa2c-c745ed170fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:55.593408 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e67de-84af-4ea2-aa2c-c745ed170fef None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:55.593692 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e67de-84af-4ea2-aa2c-c745ed170fef None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:55.593967 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e67de-84af-4ea2-aa2c-c745ed170fef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:55.594784 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e67de-84af-4ea2-aa2c-c745ed170fef None admin] RBAC: Adding query filter params (role.id=759d845eb6284e42931b4265af448ebc, scope.domain.id=default, user.id=9c29995c0b564cd7879c58c75dc099cb) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:55.595124 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e67de-84af-4ea2-aa2c-c745ed170fef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:55.597248 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e67de-84af-4ea2-aa2c-c745ed170fef None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:55.603962 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 11/20] 10.222.0.50 () {56 vars in 1322 bytes} [Fri May 27 14:59:55 2022] GET /identity/v3/role_assignments?user.id=9c29995c0b564cd7879c58c75dc099cb&scope.domain.id=default&role.id=759d845eb6284e42931b4265af448ebc => generated 233 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 14:59:55.618047 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2501493-abee-4b0d-bc9c-2e0ad3c75d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:55.618743 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2501493-abee-4b0d-bc9c-2e0ad3c75d89 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:55.619064 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2501493-abee-4b0d-bc9c-2e0ad3c75d89 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:55.619347 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2501493-abee-4b0d-bc9c-2e0ad3c75d89 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:55.620096 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2501493-abee-4b0d-bc9c-2e0ad3c75d89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:55.620408 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2501493-abee-4b0d-bc9c-2e0ad3c75d89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:55.621842 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2501493-abee-4b0d-bc9c-2e0ad3c75d89 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:55.641534 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 10/21] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 14:59:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:58.336390 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4602ea3c-3774-42e9-b9f9-4f04de0e1e9e None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:58.337524 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4602ea3c-3774-42e9-b9f9-4f04de0e1e9e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:58.337950 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4602ea3c-3774-42e9-b9f9-4f04de0e1e9e None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:58.339489 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 12/22] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 14:59:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:59:58.344885 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b8393-b8ff-4edd-9ed0-a512db0d099f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:58.346181 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b8393-b8ff-4edd-9ed0-a512db0d099f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:58.346627 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2b8393-b8ff-4edd-9ed0-a512db0d099f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:58.371841 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3b2b8393-b8ff-4edd-9ed0-a512db0d099f None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:58.373269 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3b2b8393-b8ff-4edd-9ed0-a512db0d099f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:58.454396 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 11/23] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 14:59:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:59:58.488702 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4602ea3c-3774-42e9-b9f9-4f04de0e1e9e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:58.621048 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4602ea3c-3774-42e9-b9f9-4f04de0e1e9e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:58.661467 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb1b38-34ed-4974-ab2f-b20c3ce25c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:58.662538 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb1b38-34ed-4974-ab2f-b20c3ce25c63 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:58.663004 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb1b38-34ed-4974-ab2f-b20c3ce25c63 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:58.663435 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb1b38-34ed-4974-ab2f-b20c3ce25c63 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:58.679884 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fb1b38-34ed-4974-ab2f-b20c3ce25c63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 14:59:58.687793 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-e2fb1b38-34ed-4974-ab2f-b20c3ce25c63 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 14:59:58.687793 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:58.687793 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:58.687793 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:58.687793 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:58.687793 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:58.687793 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:58.687793 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:58.687793 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:58.687793 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:58.687793 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:58.687793 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:58.687793 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 14:59:58.687793 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 14:59:58.687793 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 14:59:58.687793 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 14:59:58.688818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 14:59:58.688818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 14:59:58.688818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 14:59:58.688818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 14:59:58.688818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:58.688818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:58.688818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:58.688818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:58.688818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:58.688818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:58.688818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:58.688818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 14:59:58.688818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:58.688818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:58.688818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:58.688818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:58.690363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:58.690363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:58.690363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:58.690363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:58.690363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 14:59:58.690363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 14:59:58.690363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 14:59:58.690363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 14:59:58.690363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 14:59:58.690363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 14:59:58.690363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 14:59:58.690363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 14:59:58.696516 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 13/24] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 14:59:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:58.703312 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2b8393-b8ff-4edd-9ed0-a512db0d099f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:58.778726 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:58.852289 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2b8393-b8ff-4edd-9ed0-a512db0d099f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:58.892817 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd92094-0a7f-4a7a-b142-5751bf291f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:58.893509 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd92094-0a7f-4a7a-b142-5751bf291f84 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:58.893798 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd92094-0a7f-4a7a-b142-5751bf291f84 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:58.894082 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd92094-0a7f-4a7a-b142-5751bf291f84 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:58.894880 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd92094-0a7f-4a7a-b142-5751bf291f84 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 14:59:58.895196 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd92094-0a7f-4a7a-b142-5751bf291f84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:58.896594 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd92094-0a7f-4a7a-b142-5751bf291f84 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:58.915827 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 12/25] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 14:59:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:58.922985 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb1b38-34ed-4974-ab2f-b20c3ce25c63 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:58.928530 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:59.056922 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb1b38-34ed-4974-ab2f-b20c3ce25c63 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:59.068231 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a827a101-757f-49c4-b890-78b82c07a8ad None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:59.068519 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a827a101-757f-49c4-b890-78b82c07a8ad None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:59.068792 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a827a101-757f-49c4-b890-78b82c07a8ad None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:59.070162 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 13/26] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 14:59:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 14:59:59.075351 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc389c10-80ba-4eea-a15a-5123d0d7db79 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:59.075634 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc389c10-80ba-4eea-a15a-5123d0d7db79 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:59.075912 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc389c10-80ba-4eea-a15a-5123d0d7db79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:59.107484 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-fc389c10-80ba-4eea-a15a-5123d0d7db79 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:59.108670 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fc389c10-80ba-4eea-a15a-5123d0d7db79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:59.138375 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a49d9e-5ce3-449f-b65c-fea68b2b2155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:59.139296 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a49d9e-5ce3-449f-b65c-fea68b2b2155 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:59.139739 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a49d9e-5ce3-449f-b65c-fea68b2b2155 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:59.140171 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a49d9e-5ce3-449f-b65c-fea68b2b2155 None admin] PATH_INFO: `/v3/users/9c29995c0b564cd7879c58c75dc099cb` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:59.141601 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a49d9e-5ce3-449f-b65c-fea68b2b2155 None admin] RBAC: Authorizing `identity:get_user(user_id=9c29995c0b564cd7879c58c75dc099cb)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:59.143179 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a49d9e-5ce3-449f-b65c-fea68b2b2155 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:59.144826 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 14/27] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 14:59:58 2022] GET /identity/v3/users/9c29995c0b564cd7879c58c75dc099cb => generated 247 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:59.151205 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a49d9e-5ce3-449f-b65c-fea68b2b2155 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:59.283663 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:59.291774 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 14/28] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 14:59:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 14:59:59.293143 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0855d-809a-4f45-b498-b83c1ce24077 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:59.293440 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0855d-809a-4f45-b498-b83c1ce24077 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:59.293723 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0855d-809a-4f45-b498-b83c1ce24077 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:59.295143 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a49d9e-5ce3-449f-b65c-fea68b2b2155 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:59.295356 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 15/29] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 14:59:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:59:59.296529 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea05be9-e6c7-4dcc-9b42-fe265d9d30c1 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:59.296823 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea05be9-e6c7-4dcc-9b42-fe265d9d30c1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:59.297119 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea05be9-e6c7-4dcc-9b42-fe265d9d30c1 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:59.299288 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 16/30] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 14:59:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:59:59.304736 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ec2b0-7c1f-499a-a7e0-47599397dec7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:59.305028 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ec2b0-7c1f-499a-a7e0-47599397dec7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:59.305313 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ec2b0-7c1f-499a-a7e0-47599397dec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:59.326423 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-563ec2b0-7c1f-499a-a7e0-47599397dec7 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:59.327978 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-563ec2b0-7c1f-499a-a7e0-47599397dec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:59.342616 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081da9f3-a218-4401-86f7-0a6b46db6cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:59.343491 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081da9f3-a218-4401-86f7-0a6b46db6cf8 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:59.343999 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081da9f3-a218-4401-86f7-0a6b46db6cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:59.344440 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081da9f3-a218-4401-86f7-0a6b46db6cf8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:59.346601 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081da9f3-a218-4401-86f7-0a6b46db6cf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:59.348385 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081da9f3-a218-4401-86f7-0a6b46db6cf8 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:59.350811 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 15/31] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 14:59:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 14:59:59.352304 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b32aae-1bb5-4a63-9846-470011c21717 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:59.352758 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b32aae-1bb5-4a63-9846-470011c21717 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:59.353195 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b32aae-1bb5-4a63-9846-470011c21717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:59.369331 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:59.373910 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-92b32aae-1bb5-4a63-9846-470011c21717 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:59.375399 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-92b32aae-1bb5-4a63-9846-470011c21717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:59.417166 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 17/32] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 14:59:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:59:59.418154 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563ec2b0-7c1f-499a-a7e0-47599397dec7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:59.470987 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 16/33] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 14:59:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 14:59:59.472070 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b32aae-1bb5-4a63-9846-470011c21717 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:59.538099 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563ec2b0-7c1f-499a-a7e0-47599397dec7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:59.551682 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:59.578413 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922553b7-736c-41d4-aa4a-74a82307c2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:59.579077 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922553b7-736c-41d4-aa4a-74a82307c2b8 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:59.579339 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922553b7-736c-41d4-aa4a-74a82307c2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:59.579741 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922553b7-736c-41d4-aa4a-74a82307c2b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:59.580565 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922553b7-736c-41d4-aa4a-74a82307c2b8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:59.581957 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922553b7-736c-41d4-aa4a-74a82307c2b8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:59.585031 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b32aae-1bb5-4a63-9846-470011c21717 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:59.623580 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67afe75f-dbd3-4c0b-aa1c-778fd7d04cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:59.624458 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67afe75f-dbd3-4c0b-aa1c-778fd7d04cde None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:59.624748 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67afe75f-dbd3-4c0b-aa1c-778fd7d04cde None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:59.625035 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67afe75f-dbd3-4c0b-aa1c-778fd7d04cde None admin] PATH_INFO: `/v3/domains/default/users/9c29995c0b564cd7879c58c75dc099cb/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:59.627900 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67afe75f-dbd3-4c0b-aa1c-778fd7d04cde None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=9c29995c0b564cd7879c58c75dc099cb, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:59.629358 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67afe75f-dbd3-4c0b-aa1c-778fd7d04cde None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:59.710905 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 18/34] 10.222.0.50 () {60 vars in 1131 bytes} [Fri May 27 14:59:59 2022] POST /identity/v3/roles => generated 219 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 14:59:59.710905 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a684844-0054-4775-a148-bfebb78b5d1f None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:59.710905 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a684844-0054-4775-a148-bfebb78b5d1f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:59.710905 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a684844-0054-4775-a148-bfebb78b5d1f None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:59.710905 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 19/35] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 14:59:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 14:59:59.710905 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a684844-0054-4775-a148-bfebb78b5d1f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:59.747356 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:59.763388 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 17/36] 10.222.0.50 () {58 vars in 1361 bytes} [Fri May 27 14:59:59 2022] PUT /identity/v3/domains/default/users/9c29995c0b564cd7879c58c75dc099cb/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 14:59:59.765317 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67afe75f-dbd3-4c0b-aa1c-778fd7d04cde None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 14:59:59.841251 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a684844-0054-4775-a148-bfebb78b5d1f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:59.881144 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3aec93a-39ac-4c80-919d-893ab7c4bbaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:59.882146 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aec93a-39ac-4c80-919d-893ab7c4bbaa None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:59.882507 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aec93a-39ac-4c80-919d-893ab7c4bbaa None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:59.882947 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aec93a-39ac-4c80-919d-893ab7c4bbaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:59.896092 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aec93a-39ac-4c80-919d-893ab7c4bbaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:59.897897 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3aec93a-39ac-4c80-919d-893ab7c4bbaa None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 14:59:59.919361 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67afe75f-dbd3-4c0b-aa1c-778fd7d04cde None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 14:59:59.925380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-f3aec93a-39ac-4c80-919d-893ab7c4bbaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:59.925380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 14:59:59.925380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 14:59:59.925380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 14:59:59.925380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 14:59:59.925380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 14:59:59.925380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 14:59:59.925380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 14:59:59.925380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 14:59:59.925380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 14:59:59.925380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 14:59:59.925380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 14:59:59.925380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 14:59:59.925380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 14:59:59.925380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 14:59:59.926286 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 14:59:59.926286 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 14:59:59.926286 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 14:59:59.926286 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 14:59:59.926286 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 14:59:59.926286 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 14:59:59.926286 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 14:59:59.926286 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 14:59:59.926286 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 14:59:59.926286 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 14:59:59.926286 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 14:59:59.926286 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 14:59:59.926286 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 14:59:59.926286 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 14:59:59.926286 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 14:59:59.926286 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 14:59:59.926286 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 14:59:59.927122 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 14:59:59.927122 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 14:59:59.927122 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 14:59:59.927122 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 14:59:59.927122 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 20/37] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 14:59:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 14:59:59.928156 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf5985-6d6f-48b1-be9e-0674b62f9a61 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:59.930905 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf5985-6d6f-48b1-be9e-0674b62f9a61 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:59.930905 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf5985-6d6f-48b1-be9e-0674b62f9a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:59.955929 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-30bf5985-6d6f-48b1-be9e-0674b62f9a61 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 14:59:59.958729 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-30bf5985-6d6f-48b1-be9e-0674b62f9a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 14:59:59.959589 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 14:59:59.968150 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6cc611-3a6d-486d-a48e-10d44892d42b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 14:59:59.969050 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6cc611-3a6d-486d-a48e-10d44892d42b None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 14:59:59.969449 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6cc611-3a6d-486d-a48e-10d44892d42b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 14:59:59.969835 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6cc611-3a6d-486d-a48e-10d44892d42b None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 14:59:59.972031 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6cc611-3a6d-486d-a48e-10d44892d42b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 14:59:59.974154 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6cc611-3a6d-486d-a48e-10d44892d42b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:00.001078 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 18/38] 10.222.0.50 () {60 vars in 1131 bytes} [Fri May 27 14:59:59 2022] POST /identity/v3/roles => generated 225 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:00:00.002703 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e0c03-2e9b-4af2-8099-9e780ffdb228 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:00.003028 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e0c03-2e9b-4af2-8099-9e780ffdb228 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:00.003314 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e0c03-2e9b-4af2-8099-9e780ffdb228 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:00.004920 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 19/39] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:00.005522 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e0c03-2e9b-4af2-8099-9e780ffdb228 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:00.063283 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 21/40] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 14:59:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:00:00.065045 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de693c2-4645-40a7-a243-dd07226d6c16 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:00.065335 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de693c2-4645-40a7-a243-dd07226d6c16 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:00.065617 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de693c2-4645-40a7-a243-dd07226d6c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:00.088881 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1de693c2-4645-40a7-a243-dd07226d6c16 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:00.090218 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1de693c2-4645-40a7-a243-dd07226d6c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:00.094035 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:00.171673 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e0c03-2e9b-4af2-8099-9e780ffdb228 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:00.200147 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 22/41] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:00:00.201286 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de693c2-4645-40a7-a243-dd07226d6c16 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:00.223172 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f27ade-3cd4-4451-a02d-fe8a687efb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:00.223869 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f27ade-3cd4-4451-a02d-fe8a687efb22 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:00.224160 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f27ade-3cd4-4451-a02d-fe8a687efb22 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:00.224445 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f27ade-3cd4-4451-a02d-fe8a687efb22 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:00.225203 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f27ade-3cd4-4451-a02d-fe8a687efb22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:00.225519 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f27ade-3cd4-4451-a02d-fe8a687efb22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:00.226982 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f27ade-3cd4-4451-a02d-fe8a687efb22 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:00.255965 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 20/42] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:00:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:00.257171 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f27ade-3cd4-4451-a02d-fe8a687efb22 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:00.328291 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:00.331031 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de693c2-4645-40a7-a243-dd07226d6c16 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:00.370922 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2c312-9790-42ae-81b5-3e8ef431f008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:00.371532 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2c312-9790-42ae-81b5-3e8ef431f008 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:00.371807 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2c312-9790-42ae-81b5-3e8ef431f008 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:00.372091 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f2c312-9790-42ae-81b5-3e8ef431f008 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:00.373272 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f2c312-9790-42ae-81b5-3e8ef431f008 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:00.375143 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f2c312-9790-42ae-81b5-3e8ef431f008 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:00.385189 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:00.386032 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f27ade-3cd4-4451-a02d-fe8a687efb22 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:00.390884 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 23/43] 10.222.0.50 () {60 vars in 1131 bytes} [Fri May 27 15:00:00 2022] POST /identity/v3/roles => generated 223 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:00:00.392599 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2c312-9790-42ae-81b5-3e8ef431f008 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:00.444465 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36fa530-c847-4124-ab7d-1767c6d15be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:00.445291 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36fa530-c847-4124-ab7d-1767c6d15be1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:00.445582 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36fa530-c847-4124-ab7d-1767c6d15be1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:00.445868 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36fa530-c847-4124-ab7d-1767c6d15be1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:00.447898 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36fa530-c847-4124-ab7d-1767c6d15be1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:00.449857 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36fa530-c847-4124-ab7d-1767c6d15be1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:00.452394 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 21/44] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:00:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:00.459436 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36fa530-c847-4124-ab7d-1767c6d15be1 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:00.531084 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f2c312-9790-42ae-81b5-3e8ef431f008 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:00.581491 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00f3f2f-97a8-493a-b2d3-94bd0ecd9145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:00.582202 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f3f2f-97a8-493a-b2d3-94bd0ecd9145 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:00.582491 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f3f2f-97a8-493a-b2d3-94bd0ecd9145 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:00.582771 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f3f2f-97a8-493a-b2d3-94bd0ecd9145 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:00.583676 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36fa530-c847-4124-ab7d-1767c6d15be1 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:00.583807 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00f3f2f-97a8-493a-b2d3-94bd0ecd9145 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:00.585234 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00f3f2f-97a8-493a-b2d3-94bd0ecd9145 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:00.608387 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:00.619441 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 24/45] 10.222.0.50 () {60 vars in 1141 bytes} [Fri May 27 15:00:00 2022] POST /identity/v3/projects => generated 301 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:00:00.631596 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e910ad-c857-4950-a4c8-20ebfc5fdcbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:00.632553 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e910ad-c857-4950-a4c8-20ebfc5fdcbb None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:00.632990 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e910ad-c857-4950-a4c8-20ebfc5fdcbb None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:00.633417 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e910ad-c857-4950-a4c8-20ebfc5fdcbb None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:00.634323 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e910ad-c857-4950-a4c8-20ebfc5fdcbb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:00.636456 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e910ad-c857-4950-a4c8-20ebfc5fdcbb None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:00.668606 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 22/46] 10.222.0.50 () {60 vars in 1141 bytes} [Fri May 27 15:00:00 2022] POST /identity/v3/projects => generated 312 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:00:02.178492 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361f72b-1519-4dc9-81a9-b48387e6d948 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:02.179022 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361f72b-1519-4dc9-81a9-b48387e6d948 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:02.179109 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a361f72b-1519-4dc9-81a9-b48387e6d948 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:02.180452 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 25/47] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:02.185723 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f939a9-1624-4261-b8ae-9992956b48bc None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:02.186106 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f939a9-1624-4261-b8ae-9992956b48bc None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:02.186288 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f939a9-1624-4261-b8ae-9992956b48bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:02.203101 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-83f939a9-1624-4261-b8ae-9992956b48bc None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:02.204634 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-83f939a9-1624-4261-b8ae-9992956b48bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:02.275437 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 23/48] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:00:02.299011 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a361f72b-1519-4dc9-81a9-b48387e6d948 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:02.352385 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:02.400519 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a361f72b-1519-4dc9-81a9-b48387e6d948 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:02.434485 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1257e6d7-92b2-4f8e-b41d-4087ffda7ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:02.435235 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1257e6d7-92b2-4f8e-b41d-4087ffda7ef3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:02.435521 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1257e6d7-92b2-4f8e-b41d-4087ffda7ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:02.435805 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1257e6d7-92b2-4f8e-b41d-4087ffda7ef3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:02.446899 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1257e6d7-92b2-4f8e-b41d-4087ffda7ef3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:02.456620 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-1257e6d7-92b2-4f8e-b41d-4087ffda7ef3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:00:02.456620 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:02.456620 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:02.456620 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:02.456620 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:02.456620 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:02.456620 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:02.456620 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:02.456620 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:02.456620 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:02.456620 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:02.456620 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:02.456620 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:02.456620 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:02.456620 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:02.456620 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:02.457435 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:02.457435 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:02.457435 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:02.457435 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:02.457435 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:02.457435 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:02.457435 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:02.457435 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:02.457435 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:02.457435 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:02.457435 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:02.457435 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:00:02.457435 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:02.457435 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:02.457435 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:02.457435 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:02.458216 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:02.458216 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:02.458216 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:02.458216 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:02.458216 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:02.458216 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:02.458216 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:02.458216 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:02.458216 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:02.458216 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:02.458216 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:00:02.458216 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:00:02.458216 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 26/49] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 15:00:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:00:02.463911 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f939a9-1624-4261-b8ae-9992956b48bc None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:02.556760 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:02.559913 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f939a9-1624-4261-b8ae-9992956b48bc None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:02.593832 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca07daa5-78d8-44a7-b15c-1e1134b35346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:02.594492 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca07daa5-78d8-44a7-b15c-1e1134b35346 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:02.594770 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca07daa5-78d8-44a7-b15c-1e1134b35346 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:02.595656 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca07daa5-78d8-44a7-b15c-1e1134b35346 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:02.596415 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca07daa5-78d8-44a7-b15c-1e1134b35346 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:02.596721 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca07daa5-78d8-44a7-b15c-1e1134b35346 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:02.598102 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca07daa5-78d8-44a7-b15c-1e1134b35346 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:02.606017 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 24/50] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:00:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:00:02.611950 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1257e6d7-92b2-4f8e-b41d-4087ffda7ef3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:02.711025 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1257e6d7-92b2-4f8e-b41d-4087ffda7ef3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:02.726658 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:02.748910 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bd2b28-ad8a-407c-84ca-92696b6cf4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:02.749795 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd2b28-ad8a-407c-84ca-92696b6cf4bc None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:02.750079 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd2b28-ad8a-407c-84ca-92696b6cf4bc None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:02.750354 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bd2b28-ad8a-407c-84ca-92696b6cf4bc None admin] PATH_INFO: `/v3/users/9c29995c0b564cd7879c58c75dc099cb` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:02.751930 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bd2b28-ad8a-407c-84ca-92696b6cf4bc None admin] RBAC: Authorizing `identity:get_user(user_id=9c29995c0b564cd7879c58c75dc099cb)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:02.753397 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bd2b28-ad8a-407c-84ca-92696b6cf4bc None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:02.755066 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 27/51] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:00:02 2022] GET /identity/v3/users/9c29995c0b564cd7879c58c75dc099cb => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:02.760406 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca07daa5-78d8-44a7-b15c-1e1134b35346 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:02.862158 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca07daa5-78d8-44a7-b15c-1e1134b35346 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:02.895307 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3924e0f3-b764-479e-8b50-8b5c44d33a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:02.896089 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924e0f3-b764-479e-8b50-8b5c44d33a0a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:02.896376 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924e0f3-b764-479e-8b50-8b5c44d33a0a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:02.896658 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3924e0f3-b764-479e-8b50-8b5c44d33a0a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:02.897817 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3924e0f3-b764-479e-8b50-8b5c44d33a0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:02.899315 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3924e0f3-b764-479e-8b50-8b5c44d33a0a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:02.900936 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 25/52] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:00:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:00:02.905320 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a18a3-20a1-4796-b787-945d16e63c4b None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:02.905633 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a18a3-20a1-4796-b787-945d16e63c4b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:02.905921 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a18a3-20a1-4796-b787-945d16e63c4b None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:02.906711 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3924e0f3-b764-479e-8b50-8b5c44d33a0a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:02.907393 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 28/53] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:02.912468 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf5aee-02a8-4cb6-9d29-d4c33612e1f2 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:02.912769 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf5aee-02a8-4cb6-9d29-d4c33612e1f2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:02.913056 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf5aee-02a8-4cb6-9d29-d4c33612e1f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:02.930301 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-02cf5aee-02a8-4cb6-9d29-d4c33612e1f2 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:02.931508 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-02cf5aee-02a8-4cb6-9d29-d4c33612e1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:02.940998 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:02.998617 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 29/54] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 15:00:03.009946 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3924e0f3-b764-479e-8b50-8b5c44d33a0a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:03.022070 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf5aee-02a8-4cb6-9d29-d4c33612e1f2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:03.042812 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b13dcfb-a285-48f0-b382-ddf5404c237e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:03.043484 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13dcfb-a285-48f0-b382-ddf5404c237e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:03.043768 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13dcfb-a285-48f0-b382-ddf5404c237e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:03.044044 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b13dcfb-a285-48f0-b382-ddf5404c237e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:03.044862 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b13dcfb-a285-48f0-b382-ddf5404c237e None admin] RBAC: Adding query filter params (role.id=759d845eb6284e42931b4265af448ebc, scope.domain.id=default, user.id=9c29995c0b564cd7879c58c75dc099cb) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:03.045187 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b13dcfb-a285-48f0-b382-ddf5404c237e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:03.046589 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b13dcfb-a285-48f0-b382-ddf5404c237e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:03.052384 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 26/55] 10.222.0.50 () {56 vars in 1322 bytes} [Fri May 27 15:00:02 2022] GET /identity/v3/role_assignments?user.id=9c29995c0b564cd7879c58c75dc099cb&scope.domain.id=default&role.id=759d845eb6284e42931b4265af448ebc => generated 530 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:03.123429 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf5aee-02a8-4cb6-9d29-d4c33612e1f2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:03.135533 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:03.163094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a840a733-df74-49e0-a550-c60585e1c361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:03.164189 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a840a733-df74-49e0-a550-c60585e1c361 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:03.164474 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a840a733-df74-49e0-a550-c60585e1c361 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:03.164744 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a840a733-df74-49e0-a550-c60585e1c361 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:03.165956 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a840a733-df74-49e0-a550-c60585e1c361 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:03.167484 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a840a733-df74-49e0-a550-c60585e1c361 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:03.169138 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 30/56] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:00:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:00:03.175375 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b13dcfb-a285-48f0-b382-ddf5404c237e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:03.274738 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:03.281732 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b13dcfb-a285-48f0-b382-ddf5404c237e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:03.315426 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41fb65d-2271-4ca0-8fb4-14ecb0f11a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:03.316094 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41fb65d-2271-4ca0-8fb4-14ecb0f11a32 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:03.316379 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41fb65d-2271-4ca0-8fb4-14ecb0f11a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:03.316659 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41fb65d-2271-4ca0-8fb4-14ecb0f11a32 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:03.317367 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41fb65d-2271-4ca0-8fb4-14ecb0f11a32 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:03.318824 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41fb65d-2271-4ca0-8fb4-14ecb0f11a32 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:03.342714 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 27/57] 10.222.0.50 () {60 vars in 1140 bytes} [Fri May 27 15:00:03 2022] POST /identity/v3/projects => generated 298 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:00:05.523536 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10dd34c-c314-47e2-8f59-4e218eeea362 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:05.524057 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10dd34c-c314-47e2-8f59-4e218eeea362 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:05.524147 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10dd34c-c314-47e2-8f59-4e218eeea362 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:05.525451 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 31/58] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:05.530506 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18b8fd-6943-4f20-9536-926d04404c4d None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:05.530928 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18b8fd-6943-4f20-9536-926d04404c4d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:05.531098 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18b8fd-6943-4f20-9536-926d04404c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:05.548294 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7e18b8fd-6943-4f20-9536-926d04404c4d None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:05.549588 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7e18b8fd-6943-4f20-9536-926d04404c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:05.613005 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 28/59] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:00:05.636731 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10dd34c-c314-47e2-8f59-4e218eeea362 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:05.733834 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10dd34c-c314-47e2-8f59-4e218eeea362 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:05.765345 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba8207-1a46-479f-8c24-bb925950510e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:05.766112 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba8207-1a46-479f-8c24-bb925950510e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:05.766391 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba8207-1a46-479f-8c24-bb925950510e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:05.766663 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba8207-1a46-479f-8c24-bb925950510e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:05.768436 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbba8207-1a46-479f-8c24-bb925950510e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:05.769856 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbba8207-1a46-479f-8c24-bb925950510e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:05.771454 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 32/60] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:00:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:05.776817 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e18b8fd-6943-4f20-9536-926d04404c4d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:05.851252 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:05.873694 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e18b8fd-6943-4f20-9536-926d04404c4d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:05.906944 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32af82f4-53ff-4da6-ab05-0d886005e1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:05.907624 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af82f4-53ff-4da6-ab05-0d886005e1e1 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:05.907907 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af82f4-53ff-4da6-ab05-0d886005e1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:05.908182 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32af82f4-53ff-4da6-ab05-0d886005e1e1 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:05.908876 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32af82f4-53ff-4da6-ab05-0d886005e1e1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:05.910312 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32af82f4-53ff-4da6-ab05-0d886005e1e1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:05.935417 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 29/61] 10.222.0.50 () {60 vars in 1132 bytes} [Fri May 27 15:00:05 2022] POST /identity/v3/users => generated 275 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:00:09.187225 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780d72e-036d-4d18-9273-ab5d4c27f671 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:09.188220 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780d72e-036d-4d18-9273-ab5d4c27f671 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:09.189301 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780d72e-036d-4d18-9273-ab5d4c27f671 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:09.191483 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 33/62] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:09.197099 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a0fd7-2607-4ed6-9a0e-ff65daa2685a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:09.197659 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a0fd7-2607-4ed6-9a0e-ff65daa2685a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:09.197659 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a0fd7-2607-4ed6-9a0e-ff65daa2685a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:09.227149 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d72a0fd7-2607-4ed6-9a0e-ff65daa2685a None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:09.228357 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d72a0fd7-2607-4ed6-9a0e-ff65daa2685a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:09.327204 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 30/63] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 15:00:09.359365 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780d72e-036d-4d18-9273-ab5d4c27f671 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:09.428488 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:09.492515 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780d72e-036d-4d18-9273-ab5d4c27f671 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:09.512478 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8262584-c84f-45f4-bea6-33897ae86d8e None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:09.512773 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8262584-c84f-45f4-bea6-33897ae86d8e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:09.513064 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8262584-c84f-45f4-bea6-33897ae86d8e None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:09.514465 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 31/64] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:09.519886 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78b64a-b506-4033-bb85-099c63a07a87 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:09.520183 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78b64a-b506-4033-bb85-099c63a07a87 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:09.520470 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb78b64a-b506-4033-bb85-099c63a07a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:09.535569 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e6df5f-665f-4b31-886f-31b0db5c9fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:09.536378 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e6df5f-665f-4b31-886f-31b0db5c9fd4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:09.536658 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e6df5f-665f-4b31-886f-31b0db5c9fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:09.536928 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e6df5f-665f-4b31-886f-31b0db5c9fd4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:09.556003 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-cb78b64a-b506-4033-bb85-099c63a07a87 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:09.557193 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cb78b64a-b506-4033-bb85-099c63a07a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:09.560118 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e6df5f-665f-4b31-886f-31b0db5c9fd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:09.568573 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-41e6df5f-665f-4b31-886f-31b0db5c9fd4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 15:00:09.568573 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:09.568573 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:09.568573 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:09.568573 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:09.568573 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:09.568573 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:09.568573 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:09.568573 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:09.568573 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:09.568573 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:09.568573 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:09.568573 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:09.568573 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:09.568573 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:09.568573 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:09.569675 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:09.569675 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:09.569675 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:09.569675 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:09.569675 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:09.569675 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:09.569675 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:09.569675 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:09.569675 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:09.569675 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:09.569675 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:09.569675 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:00:09.569675 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:09.569675 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:09.569675 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:09.569675 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:09.570484 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:09.570484 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:09.570484 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:09.570484 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:09.570484 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:09.570484 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:09.570484 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:09.570484 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:09.570484 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:09.570484 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:09.570484 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 15:00:09.570484 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:00:09.570484 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 34/65] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:00:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:00:09.585439 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e6df5f-665f-4b31-886f-31b0db5c9fd4 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:09.691511 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:09.712806 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 32/66] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:00:09.737792 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb78b64a-b506-4033-bb85-099c63a07a87 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:09.801012 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e6df5f-665f-4b31-886f-31b0db5c9fd4 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:09.810623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:09.855038 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e24c1f-c002-4bac-8965-9463c7cc9e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:09.855954 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e24c1f-c002-4bac-8965-9463c7cc9e5c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:09.856441 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e24c1f-c002-4bac-8965-9463c7cc9e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:09.856731 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e24c1f-c002-4bac-8965-9463c7cc9e5c None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:09.857491 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e24c1f-c002-4bac-8965-9463c7cc9e5c None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:09.857757 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb78b64a-b506-4033-bb85-099c63a07a87 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:09.857856 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e24c1f-c002-4bac-8965-9463c7cc9e5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:09.859214 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e24c1f-c002-4bac-8965-9463c7cc9e5c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:09.868641 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 35/67] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 15:00:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:09.870007 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737653b-f878-4ab5-bf45-2484f2e9638e None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:09.870308 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737653b-f878-4ab5-bf45-2484f2e9638e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:09.870593 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737653b-f878-4ab5-bf45-2484f2e9638e None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:09.872127 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 36/68] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:09.873231 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7694b269-3aea-4e9c-953a-c59247c63832 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:09.873521 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7694b269-3aea-4e9c-953a-c59247c63832 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:09.873803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7694b269-3aea-4e9c-953a-c59247c63832 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:09.876203 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 37/69] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:09 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:09.876203 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7694b269-3aea-4e9c-953a-c59247c63832 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:09.895900 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35bef1a-be0c-4332-86d1-ce5b8451ad45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:09.896699 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35bef1a-be0c-4332-86d1-ce5b8451ad45 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:09.896987 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35bef1a-be0c-4332-86d1-ce5b8451ad45 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:09.897266 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35bef1a-be0c-4332-86d1-ce5b8451ad45 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:09.909381 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35bef1a-be0c-4332-86d1-ce5b8451ad45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:09.917273 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-e35bef1a-be0c-4332-86d1-ce5b8451ad45 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:00:09.917273 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:09.917273 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:09.917273 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:09.917273 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:09.917273 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:09.917273 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:09.917273 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:09.917273 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:09.917273 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:09.917273 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:09.917273 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:09.917273 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:09.917273 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:09.917273 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:09.917273 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:09.918157 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:09.918157 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:09.918157 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:09.918157 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:09.918157 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:09.918157 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:09.918157 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:09.918157 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:09.918157 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:09.918157 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:09.918157 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:09.918157 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:00:09.918157 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:09.918157 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:09.918157 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:09.918157 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:09.918949 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:09.918949 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:09.918949 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:09.918949 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:09.918949 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:09.918949 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:09.918949 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:09.918949 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:09.918949 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:09.918949 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:09.918949 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:00:09.918949 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:00:09.918949 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 33/70] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 15:00:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:00:09.919822 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211b750-aa9b-4e4e-962a-b3797099575d None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:09.920116 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211b750-aa9b-4e4e-962a-b3797099575d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:09.920403 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211b750-aa9b-4e4e-962a-b3797099575d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:09.937149 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a211b750-aa9b-4e4e-962a-b3797099575d None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:09.938342 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a211b750-aa9b-4e4e-962a-b3797099575d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:09.982939 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7694b269-3aea-4e9c-953a-c59247c63832 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:09.985426 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:10.005582 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 34/71] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:00:10.006942 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acb6f2-04bd-4da7-893c-3d41bd0f4904 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:10.007236 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acb6f2-04bd-4da7-893c-3d41bd0f4904 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:10.007515 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acb6f2-04bd-4da7-893c-3d41bd0f4904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:10.015685 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ec34ad-c1b0-46a9-8ea7-2d225122504d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:10.016617 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec34ad-c1b0-46a9-8ea7-2d225122504d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:10.017396 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec34ad-c1b0-46a9-8ea7-2d225122504d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:10.017732 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ec34ad-c1b0-46a9-8ea7-2d225122504d None admin] PATH_INFO: `/v3/users/aac9cdff3dc345eaa75a0bec09d65b4c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:10.024819 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-05acb6f2-04bd-4da7-893c-3d41bd0f4904 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:10.025992 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-05acb6f2-04bd-4da7-893c-3d41bd0f4904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:10.034629 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ec34ad-c1b0-46a9-8ea7-2d225122504d None admin] RBAC: Authorizing `identity:get_user(user_id=aac9cdff3dc345eaa75a0bec09d65b4c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:10.036138 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ec34ad-c1b0-46a9-8ea7-2d225122504d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:10.037848 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 38/72] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:00:09 2022] GET /identity/v3/users/aac9cdff3dc345eaa75a0bec09d65b4c => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:10.039105 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6dcbdb-5ed2-4151-ac85-41bb633b176b None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:10.039397 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6dcbdb-5ed2-4151-ac85-41bb633b176b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:10.039676 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6dcbdb-5ed2-4151-ac85-41bb633b176b None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:10.042887 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 39/73] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:10.042887 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6dcbdb-5ed2-4151-ac85-41bb633b176b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:10.068376 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:10.097618 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 35/74] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:00:10.098537 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05acb6f2-04bd-4da7-893c-3d41bd0f4904 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:10.143092 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6dcbdb-5ed2-4151-ac85-41bb633b176b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:10.177446 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7676467-6178-4e9c-9a42-6e81f4bb3bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:10.178142 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7676467-6178-4e9c-9a42-6e81f4bb3bda None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:10.178427 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7676467-6178-4e9c-9a42-6e81f4bb3bda None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:10.178705 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7676467-6178-4e9c-9a42-6e81f4bb3bda None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:10.179480 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7676467-6178-4e9c-9a42-6e81f4bb3bda None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:10.179783 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7676467-6178-4e9c-9a42-6e81f4bb3bda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:10.181150 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7676467-6178-4e9c-9a42-6e81f4bb3bda None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:10.189076 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 40/75] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:00:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:10.189875 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7676467-6178-4e9c-9a42-6e81f4bb3bda None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:10.200586 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05acb6f2-04bd-4da7-893c-3d41bd0f4904 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:10.214367 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:10.236278 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b1b425-72f1-4998-acd2-047e94b72efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:10.237036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1b425-72f1-4998-acd2-047e94b72efc None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:10.237324 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1b425-72f1-4998-acd2-047e94b72efc None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:10.237603 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b1b425-72f1-4998-acd2-047e94b72efc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:10.247368 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b1b425-72f1-4998-acd2-047e94b72efc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:10.255679 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-36b1b425-72f1-4998-acd2-047e94b72efc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 15:00:10.255679 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:10.255679 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:10.255679 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:10.255679 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:10.255679 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:10.255679 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:10.255679 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:10.255679 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:10.255679 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:10.255679 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:10.255679 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:10.255679 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:10.255679 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:10.255679 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:10.255679 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:10.256538 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:10.256538 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:10.256538 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:10.256538 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:10.256538 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:10.256538 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:10.256538 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:10.256538 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:10.256538 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:10.256538 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:10.256538 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:10.256538 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:00:10.256538 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:10.256538 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:10.256538 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:10.256538 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:10.257289 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:10.257289 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:10.257289 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:10.257289 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:10.257289 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:10.257289 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:10.257289 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:10.257289 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:10.257289 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:10.257289 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:10.257289 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 15:00:10.257289 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:00:10.257289 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 36/76] 10.222.0.50 () {56 vars in 1114 bytes} [Fri May 27 15:00:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:00:10.258106 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d59c1b-c6c7-4cf2-b988-4284cbf1e249 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:10.258371 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d59c1b-c6c7-4cf2-b988-4284cbf1e249 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:10.258656 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d59c1b-c6c7-4cf2-b988-4284cbf1e249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:10.275320 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b7d59c1b-c6c7-4cf2-b988-4284cbf1e249 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:10.276488 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b7d59c1b-c6c7-4cf2-b988-4284cbf1e249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:10.281831 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:10.294574 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7676467-6178-4e9c-9a42-6e81f4bb3bda None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:10.328863 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed49c4-3bc3-4a46-a884-602c4bb3b284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:10.330371 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed49c4-3bc3-4a46-a884-602c4bb3b284 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:10.330798 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed49c4-3bc3-4a46-a884-602c4bb3b284 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:10.331108 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed49c4-3bc3-4a46-a884-602c4bb3b284 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:10.342505 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 37/77] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:00:10.343376 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d59c1b-c6c7-4cf2-b988-4284cbf1e249 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:10.399004 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed49c4-3bc3-4a46-a884-602c4bb3b284 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:10.400466 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed49c4-3bc3-4a46-a884-602c4bb3b284 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:10.448474 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d59c1b-c6c7-4cf2-b988-4284cbf1e249 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:10.449942 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:10.483278 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f630c0-a822-4c23-800b-96c522fd8e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:10.484066 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f630c0-a822-4c23-800b-96c522fd8e42 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:10.484351 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f630c0-a822-4c23-800b-96c522fd8e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:10.484628 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f630c0-a822-4c23-800b-96c522fd8e42 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:10.485092 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:10.494747 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f630c0-a822-4c23-800b-96c522fd8e42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:10.498328 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 41/78] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:00:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 15:00:10.499188 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed49c4-3bc3-4a46-a884-602c4bb3b284 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:10.503278 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-56f630c0-a822-4c23-800b-96c522fd8e42 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 15:00:10.503278 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:10.503278 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:10.503278 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:10.503278 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:10.503278 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:10.503278 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:10.503278 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:10.503278 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:10.503278 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:10.503278 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:10.503278 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:10.503278 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:10.503278 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:10.503278 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:10.503278 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:10.504267 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:10.504267 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:10.504267 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:10.504267 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:10.504267 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:10.504267 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:10.504267 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:10.504267 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:10.504267 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:10.504267 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:10.504267 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:10.504267 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:00:10.504267 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:10.504267 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:10.504267 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:10.504267 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:10.505036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:10.505036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:10.505036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:10.505036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:10.505036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:10.505036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:10.505036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:10.505036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:10.505036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:10.505036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:10.505036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 15:00:10.505036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:00:10.505036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 38/79] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:00:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:00:10.505036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f630c0-a822-4c23-800b-96c522fd8e42 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:10.611542 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed49c4-3bc3-4a46-a884-602c4bb3b284 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:10.615465 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f630c0-a822-4c23-800b-96c522fd8e42 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:10.651835 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50384f9e-83c9-4ef2-9271-ba3aa8e71414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:10.652631 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50384f9e-83c9-4ef2-9271-ba3aa8e71414 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:10.652993 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50384f9e-83c9-4ef2-9271-ba3aa8e71414 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:10.653383 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50384f9e-83c9-4ef2-9271-ba3aa8e71414 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:10.653984 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fe3192-6856-44f9-b6b1-b7aeacb884c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:10.654325 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50384f9e-83c9-4ef2-9271-ba3aa8e71414 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:10.654644 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50384f9e-83c9-4ef2-9271-ba3aa8e71414 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:10.654874 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe3192-6856-44f9-b6b1-b7aeacb884c9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:10.655171 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe3192-6856-44f9-b6b1-b7aeacb884c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:10.655445 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe3192-6856-44f9-b6b1-b7aeacb884c9 None admin] PATH_INFO: `/v3/users/9c29995c0b564cd7879c58c75dc099cb` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:10.656086 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50384f9e-83c9-4ef2-9271-ba3aa8e71414 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:10.656638 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fe3192-6856-44f9-b6b1-b7aeacb884c9 None admin] RBAC: Authorizing `identity:get_user(user_id=9c29995c0b564cd7879c58c75dc099cb)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:10.658659 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fe3192-6856-44f9-b6b1-b7aeacb884c9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:10.664713 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 42/80] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:00:10 2022] GET /identity/v3/users/9c29995c0b564cd7879c58c75dc099cb => generated 247 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:10.664713 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fe3192-6856-44f9-b6b1-b7aeacb884c9 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:10.664903 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 39/81] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:00:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:10.665747 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50384f9e-83c9-4ef2-9271-ba3aa8e71414 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:10.693380 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:10.729232 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:10.777730 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50384f9e-83c9-4ef2-9271-ba3aa8e71414 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:10.778253 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fe3192-6856-44f9-b6b1-b7aeacb884c9 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:10.814540 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f940f-0dd4-4238-b8e1-8a7938a16fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:10.815447 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f940f-0dd4-4238-b8e1-8a7938a16fd5 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:10.815681 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f097ecaa-4a51-4c83-8c02-0b05f511abdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:10.815787 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f940f-0dd4-4238-b8e1-8a7938a16fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:10.816006 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f940f-0dd4-4238-b8e1-8a7938a16fd5 None admin] PATH_INFO: `/v3/projects/00203c7cc02341ee8ee7a591095f0c7d` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:10.816486 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f097ecaa-4a51-4c83-8c02-0b05f511abdb None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:10.816777 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f097ecaa-4a51-4c83-8c02-0b05f511abdb None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:10.817057 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f097ecaa-4a51-4c83-8c02-0b05f511abdb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:10.817326 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5f940f-0dd4-4238-b8e1-8a7938a16fd5 None admin] RBAC: Authorizing `identity:get_project(project_id=00203c7cc02341ee8ee7a591095f0c7d)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:10.818245 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f097ecaa-4a51-4c83-8c02-0b05f511abdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:10.818916 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5f940f-0dd4-4238-b8e1-8a7938a16fd5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:10.819713 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f097ecaa-4a51-4c83-8c02-0b05f511abdb None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:10.820711 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 40/82] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:00:10 2022] GET /identity/v3/projects/00203c7cc02341ee8ee7a591095f0c7d => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:10.821485 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f940f-0dd4-4238-b8e1-8a7938a16fd5 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:10.821810 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 43/83] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:00:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:10.822448 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f097ecaa-4a51-4c83-8c02-0b05f511abdb None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:10.919558 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:10.926044 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f097ecaa-4a51-4c83-8c02-0b05f511abdb None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:10.929186 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f940f-0dd4-4238-b8e1-8a7938a16fd5 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:10.949477 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:10.960678 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c1437-8df3-43ff-9089-72f3bd3ae555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:10.961332 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c1437-8df3-43ff-9089-72f3bd3ae555 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:10.961630 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c1437-8df3-43ff-9089-72f3bd3ae555 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:10.961904 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c1437-8df3-43ff-9089-72f3bd3ae555 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:10.963390 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0608063f-47fe-4f12-a0ae-3b0a3556222f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:10.964054 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608063f-47fe-4f12-a0ae-3b0a3556222f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:10.964340 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608063f-47fe-4f12-a0ae-3b0a3556222f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:10.964716 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0608063f-47fe-4f12-a0ae-3b0a3556222f None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:10.965363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0608063f-47fe-4f12-a0ae-3b0a3556222f None admin] RBAC: Adding query filter params (name=member) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:10.965677 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0608063f-47fe-4f12-a0ae-3b0a3556222f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:10.967088 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0608063f-47fe-4f12-a0ae-3b0a3556222f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:10.975152 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 41/84] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 15:00:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:10.975947 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0608063f-47fe-4f12-a0ae-3b0a3556222f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:11.035646 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c1437-8df3-43ff-9089-72f3bd3ae555 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:11.037031 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c1437-8df3-43ff-9089-72f3bd3ae555 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:11.077450 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0608063f-47fe-4f12-a0ae-3b0a3556222f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:11.111427 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46377c72-1787-436b-929f-463cad3aadf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:11.112454 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46377c72-1787-436b-929f-463cad3aadf4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:11.112734 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46377c72-1787-436b-929f-463cad3aadf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:11.113009 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46377c72-1787-436b-929f-463cad3aadf4 None admin] PATH_INFO: `/v3/users/aac9cdff3dc345eaa75a0bec09d65b4c` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:11.114199 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46377c72-1787-436b-929f-463cad3aadf4 None admin] RBAC: Authorizing `identity:get_user(user_id=aac9cdff3dc345eaa75a0bec09d65b4c)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:11.115656 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46377c72-1787-436b-929f-463cad3aadf4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:11.117434 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 42/85] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:00:10 2022] GET /identity/v3/users/aac9cdff3dc345eaa75a0bec09d65b4c => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:11.118036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46377c72-1787-436b-929f-463cad3aadf4 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:11.138761 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 44/86] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:00:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 15:00:11.139793 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c1437-8df3-43ff-9089-72f3bd3ae555 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:11.145350 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:11.167543 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:11.217086 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46377c72-1787-436b-929f-463cad3aadf4 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:11.241200 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c1437-8df3-43ff-9089-72f3bd3ae555 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:11.249850 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ab691-e359-4b2f-b114-df6a568975e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:11.250510 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ab691-e359-4b2f-b114-df6a568975e3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:11.250800 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ab691-e359-4b2f-b114-df6a568975e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:11.251139 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ab691-e359-4b2f-b114-df6a568975e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:11.251951 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7ab691-e359-4b2f-b114-df6a568975e3 None admin] RBAC: Adding query filter params (role.id=8442c42e847a4b38b76f3a7fbe9ada85, scope.project.id=00203c7cc02341ee8ee7a591095f0c7d, user.id=aac9cdff3dc345eaa75a0bec09d65b4c) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:11.252267 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7ab691-e359-4b2f-b114-df6a568975e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:11.253718 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7ab691-e359-4b2f-b114-df6a568975e3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:11.259116 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 43/87] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:00:11 2022] GET /identity/v3/role_assignments?user.id=aac9cdff3dc345eaa75a0bec09d65b4c&scope.project.id=00203c7cc02341ee8ee7a591095f0c7d&role.id=8442c42e847a4b38b76f3a7fbe9ada85 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:11.259900 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ab691-e359-4b2f-b114-df6a568975e3 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:11.277462 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0168d-3938-4f87-8a01-11203660e4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:11.278158 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0168d-3938-4f87-8a01-11203660e4cd None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:11.278440 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0168d-3938-4f87-8a01-11203660e4cd None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:11.278713 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0168d-3938-4f87-8a01-11203660e4cd None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:11.279470 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac0168d-3938-4f87-8a01-11203660e4cd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:11.280960 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac0168d-3938-4f87-8a01-11203660e4cd None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:11.310228 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 45/88] 10.222.0.50 () {60 vars in 1132 bytes} [Fri May 27 15:00:11 2022] POST /identity/v3/users => generated 289 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:00:11.316202 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0168d-3938-4f87-8a01-11203660e4cd None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:11.383271 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:11.409017 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ab691-e359-4b2f-b114-df6a568975e3 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:11.452506 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:11.469061 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf8bfac-7c47-44a1-88b6-8322b79c3d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:11.471474 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8bfac-7c47-44a1-88b6-8322b79c3d5d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:11.471878 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8bfac-7c47-44a1-88b6-8322b79c3d5d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:11.472243 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8bfac-7c47-44a1-88b6-8322b79c3d5d None admin] PATH_INFO: `/v3/users/aac9cdff3dc345eaa75a0bec09d65b4c` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:11.473620 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf8bfac-7c47-44a1-88b6-8322b79c3d5d None admin] RBAC: Authorizing `identity:get_user(user_id=aac9cdff3dc345eaa75a0bec09d65b4c)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:11.475169 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf8bfac-7c47-44a1-88b6-8322b79c3d5d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:11.477096 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 44/89] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:00:11 2022] GET /identity/v3/users/aac9cdff3dc345eaa75a0bec09d65b4c => generated 275 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:11.477732 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf8bfac-7c47-44a1-88b6-8322b79c3d5d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:11.483120 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0168d-3938-4f87-8a01-11203660e4cd None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:11.533505 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c28dc-fdfe-4512-be4a-70ff29ee57e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:11.534604 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c28dc-fdfe-4512-be4a-70ff29ee57e0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:11.535027 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c28dc-fdfe-4512-be4a-70ff29ee57e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:11.535317 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c28dc-fdfe-4512-be4a-70ff29ee57e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:11.599451 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf8bfac-7c47-44a1-88b6-8322b79c3d5d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:11.627607 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c28dc-fdfe-4512-be4a-70ff29ee57e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:11.629034 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c28dc-fdfe-4512-be4a-70ff29ee57e0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:11.642522 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4e6d4f-23b2-4ca1-8287-26f45aa11e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:11.643587 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e6d4f-23b2-4ca1-8287-26f45aa11e1b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:11.643876 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e6d4f-23b2-4ca1-8287-26f45aa11e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:11.644162 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e6d4f-23b2-4ca1-8287-26f45aa11e1b None admin] PATH_INFO: `/v3/projects/00203c7cc02341ee8ee7a591095f0c7d` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:11.646870 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4e6d4f-23b2-4ca1-8287-26f45aa11e1b None admin] RBAC: Authorizing `identity:get_project(project_id=00203c7cc02341ee8ee7a591095f0c7d)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:11.648648 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4e6d4f-23b2-4ca1-8287-26f45aa11e1b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:11.650602 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 45/90] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:00:11 2022] GET /identity/v3/projects/00203c7cc02341ee8ee7a591095f0c7d => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:11.654309 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4e6d4f-23b2-4ca1-8287-26f45aa11e1b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:11.688037 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:11.746781 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:11.777585 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 46/91] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:00:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 15:00:11.778770 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c28dc-fdfe-4512-be4a-70ff29ee57e0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:11.793571 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4e6d4f-23b2-4ca1-8287-26f45aa11e1b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:11.833375 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834508c5-7108-44bc-b518-39f346942521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:11.834330 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834508c5-7108-44bc-b518-39f346942521 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:11.834699 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834508c5-7108-44bc-b518-39f346942521 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:11.835027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834508c5-7108-44bc-b518-39f346942521 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:11.912614 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c28dc-fdfe-4512-be4a-70ff29ee57e0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:11.927728 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834508c5-7108-44bc-b518-39f346942521 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:11.930353 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834508c5-7108-44bc-b518-39f346942521 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:11.949894 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165213de-6ec8-43ab-a43e-22870c85b195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:11.950461 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165213de-6ec8-43ab-a43e-22870c85b195 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:11.950920 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165213de-6ec8-43ab-a43e-22870c85b195 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:11.951203 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165213de-6ec8-43ab-a43e-22870c85b195 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:11.952033 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165213de-6ec8-43ab-a43e-22870c85b195 None admin] RBAC: Adding query filter params (role.id=759d845eb6284e42931b4265af448ebc, scope.project.id=00203c7cc02341ee8ee7a591095f0c7d, user.id=9c29995c0b564cd7879c58c75dc099cb) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:11.952347 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165213de-6ec8-43ab-a43e-22870c85b195 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:11.953789 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165213de-6ec8-43ab-a43e-22870c85b195 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:11.963409 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:11.964015 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 47/92] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:00:11 2022] GET /identity/v3/role_assignments?user.id=9c29995c0b564cd7879c58c75dc099cb&scope.project.id=00203c7cc02341ee8ee7a591095f0c7d&role.id=759d845eb6284e42931b4265af448ebc => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:11.965043 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165213de-6ec8-43ab-a43e-22870c85b195 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:12.053484 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:12.119092 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 46/93] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:00:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 15:00:12.124777 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834508c5-7108-44bc-b518-39f346942521 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:12.135876 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165213de-6ec8-43ab-a43e-22870c85b195 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:12.201054 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb299a-ccae-4f51-b4d7-e190076d474c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:12.201868 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb299a-ccae-4f51-b4d7-e190076d474c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:12.202153 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb299a-ccae-4f51-b4d7-e190076d474c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:12.202429 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb299a-ccae-4f51-b4d7-e190076d474c None admin] PATH_INFO: `/v3/projects/00203c7cc02341ee8ee7a591095f0c7d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:12.203682 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fb299a-ccae-4f51-b4d7-e190076d474c None admin] RBAC: Authorizing `identity:get_project(project_id=00203c7cc02341ee8ee7a591095f0c7d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:12.205706 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fb299a-ccae-4f51-b4d7-e190076d474c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:12.207437 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 48/94] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:00:11 2022] GET /identity/v3/projects/00203c7cc02341ee8ee7a591095f0c7d => generated 298 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:12.215076 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb299a-ccae-4f51-b4d7-e190076d474c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:12.294174 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834508c5-7108-44bc-b518-39f346942521 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:12.331453 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:12.356906 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0468863f-5ebb-4a4c-a231-7af9b271b69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:12.357837 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0468863f-5ebb-4a4c-a231-7af9b271b69c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:12.358130 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0468863f-5ebb-4a4c-a231-7af9b271b69c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:12.358411 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0468863f-5ebb-4a4c-a231-7af9b271b69c None admin] PATH_INFO: `/v3/projects/a8102aa3c6634ff6b42c5df957f24182` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:12.363584 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0468863f-5ebb-4a4c-a231-7af9b271b69c None admin] RBAC: Authorizing `identity:get_project(project_id=a8102aa3c6634ff6b42c5df957f24182)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:12.365076 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0468863f-5ebb-4a4c-a231-7af9b271b69c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:12.367161 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 47/95] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:00:12 2022] GET /identity/v3/projects/a8102aa3c6634ff6b42c5df957f24182 => generated 312 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:12.372560 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0468863f-5ebb-4a4c-a231-7af9b271b69c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:12.461483 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb299a-ccae-4f51-b4d7-e190076d474c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:12.491009 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:12.521882 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfdab5c-2814-4e3d-8661-ec6856117530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:12.522897 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfdab5c-2814-4e3d-8661-ec6856117530 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:12.523377 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfdab5c-2814-4e3d-8661-ec6856117530 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:12.523871 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfdab5c-2814-4e3d-8661-ec6856117530 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:12.524891 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfdab5c-2814-4e3d-8661-ec6856117530 None admin] RBAC: Adding query filter params (role.id=ad9498a5e2064a468acc151fb2bce49e, scope.project.id=00203c7cc02341ee8ee7a591095f0c7d, user.id=aac9cdff3dc345eaa75a0bec09d65b4c) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:12.525407 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfdab5c-2814-4e3d-8661-ec6856117530 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:12.527088 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfdab5c-2814-4e3d-8661-ec6856117530 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:12.538875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0468863f-5ebb-4a4c-a231-7af9b271b69c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:12.556693 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 49/96] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:00:12 2022] GET /identity/v3/role_assignments?user.id=aac9cdff3dc345eaa75a0bec09d65b4c&scope.project.id=00203c7cc02341ee8ee7a591095f0c7d&role.id=ad9498a5e2064a468acc151fb2bce49e => generated 259 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:12.579781 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a27c45-4642-477b-8569-83871631175c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:12.580598 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a27c45-4642-477b-8569-83871631175c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:12.580886 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a27c45-4642-477b-8569-83871631175c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:12.581171 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a27c45-4642-477b-8569-83871631175c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:12.582011 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a27c45-4642-477b-8569-83871631175c None admin] RBAC: Adding query filter params (role.id=8442c42e847a4b38b76f3a7fbe9ada85, scope.project.id=a8102aa3c6634ff6b42c5df957f24182, user.id=aac9cdff3dc345eaa75a0bec09d65b4c) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:12.582329 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a27c45-4642-477b-8569-83871631175c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:12.583840 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a27c45-4642-477b-8569-83871631175c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:12.590703 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 48/97] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:00:12 2022] GET /identity/v3/role_assignments?user.id=aac9cdff3dc345eaa75a0bec09d65b4c&scope.project.id=a8102aa3c6634ff6b42c5df957f24182&role.id=8442c42e847a4b38b76f3a7fbe9ada85 => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:00:14.366277 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef10b6-4695-4736-959a-3a75ce9d03d7 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:14.367320 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef10b6-4695-4736-959a-3a75ce9d03d7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:14.367807 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef10b6-4695-4736-959a-3a75ce9d03d7 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:14.369400 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 50/98] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:14.374775 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7871dd-13a0-4911-8139-33aea8bcc3ce None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:14.375604 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7871dd-13a0-4911-8139-33aea8bcc3ce None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:14.376642 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7871dd-13a0-4911-8139-33aea8bcc3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:14.379560 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1626bfe3-265d-4f8e-a24f-b88a49d67ce5 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:14.380060 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1626bfe3-265d-4f8e-a24f-b88a49d67ce5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:14.380554 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1626bfe3-265d-4f8e-a24f-b88a49d67ce5 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:14.382038 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 51/99] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:14.387093 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76819e5-71ac-4310-a1cf-544c7b6c00c1 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:14.387605 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76819e5-71ac-4310-a1cf-544c7b6c00c1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:14.388102 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76819e5-71ac-4310-a1cf-544c7b6c00c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:14.399854 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2b7871dd-13a0-4911-8139-33aea8bcc3ce None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:14.401196 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2b7871dd-13a0-4911-8139-33aea8bcc3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:14.406327 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d76819e5-71ac-4310-a1cf-544c7b6c00c1 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:14.408048 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d76819e5-71ac-4310-a1cf-544c7b6c00c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:14.482875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:14.515649 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 49/100] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:00:14.528222 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 52/101] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 15:00:14.557223 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7871dd-13a0-4911-8139-33aea8bcc3ce None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:14.565316 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76819e5-71ac-4310-a1cf-544c7b6c00c1 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:14.731109 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76819e5-71ac-4310-a1cf-544c7b6c00c1 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:14.745944 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:14.764886 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7871dd-13a0-4911-8139-33aea8bcc3ce None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:14.771978 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8b80ac-be7d-45ae-b13e-6ad771a0d4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:14.772784 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b80ac-be7d-45ae-b13e-6ad771a0d4c4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:14.773076 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b80ac-be7d-45ae-b13e-6ad771a0d4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:14.773356 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b80ac-be7d-45ae-b13e-6ad771a0d4c4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:14.785278 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8b80ac-be7d-45ae-b13e-6ad771a0d4c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:14.792979 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-ad8b80ac-be7d-45ae-b13e-6ad771a0d4c4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 15:00:14.792979 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:14.792979 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:14.792979 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:14.792979 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:14.792979 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:14.792979 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:14.792979 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:14.792979 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:14.792979 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:14.792979 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:14.792979 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:14.792979 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:14.792979 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:14.792979 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:14.792979 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:14.793825 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:14.793825 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:14.793825 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:14.793825 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:14.793825 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:14.793825 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:14.793825 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:14.793825 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:14.793825 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:14.793825 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:14.793825 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:14.793825 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:00:14.793825 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:14.793825 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:14.793825 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:14.793825 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:14.794577 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:14.794577 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:14.794577 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:14.794577 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:14.794577 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:14.794577 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:14.794577 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:14.794577 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:14.794577 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:14.794577 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:14.794577 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 15:00:14.794577 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:00:14.794577 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 53/102] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:00:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:00:14.800838 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8b80ac-be7d-45ae-b13e-6ad771a0d4c4 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:14.831187 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6db6b59-bced-429a-919d-46fef93b5751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:14.832041 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db6b59-bced-429a-919d-46fef93b5751 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:14.832331 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db6b59-bced-429a-919d-46fef93b5751 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:14.832613 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db6b59-bced-429a-919d-46fef93b5751 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:14.856646 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6db6b59-bced-429a-919d-46fef93b5751 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:14.871301 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-d6db6b59-bced-429a-919d-46fef93b5751 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 15:00:14.871301 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:14.871301 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:14.871301 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:14.871301 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:14.871301 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:14.871301 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:14.871301 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:14.871301 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:14.871301 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:14.871301 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:14.871301 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:14.871301 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:14.871301 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:14.871301 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:14.871301 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:14.872234 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:14.872234 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:14.872234 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:14.872234 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:14.872234 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:14.872234 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:14.872234 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:14.872234 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:14.872234 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:14.872234 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:14.872234 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:14.872234 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:00:14.872234 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:14.872234 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:14.872234 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:14.872234 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:14.874341 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:14.874341 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:14.874341 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:14.874341 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:14.874341 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:14.874341 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:14.874341 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:14.874341 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:14.874341 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:14.874341 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:14.874341 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 15:00:14.874341 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:00:14.874341 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 50/103] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:00:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:00:14.881923 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6db6b59-bced-429a-919d-46fef93b5751 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:14.968895 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8b80ac-be7d-45ae-b13e-6ad771a0d4c4 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:14.973064 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:15.045092 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6db6b59-bced-429a-919d-46fef93b5751 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:15.055569 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2618be-e343-481a-bc10-6eef406e3590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:15.056262 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2618be-e343-481a-bc10-6eef406e3590 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:15.056552 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2618be-e343-481a-bc10-6eef406e3590 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:15.056835 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2618be-e343-481a-bc10-6eef406e3590 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:15.057803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2618be-e343-481a-bc10-6eef406e3590 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:15.057907 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2618be-e343-481a-bc10-6eef406e3590 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:15.059300 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2618be-e343-481a-bc10-6eef406e3590 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:15.073289 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 54/104] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 15:00:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:00:15.078731 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2618be-e343-481a-bc10-6eef406e3590 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:15.083079 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b907c30-0de7-4c6a-b9fc-442f34a042a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:15.084009 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b907c30-0de7-4c6a-b9fc-442f34a042a6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:15.084494 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b907c30-0de7-4c6a-b9fc-442f34a042a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:15.084993 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b907c30-0de7-4c6a-b9fc-442f34a042a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:15.086213 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b907c30-0de7-4c6a-b9fc-442f34a042a6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:15.086722 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b907c30-0de7-4c6a-b9fc-442f34a042a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:15.088593 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b907c30-0de7-4c6a-b9fc-442f34a042a6 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:15.098210 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 51/105] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 15:00:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:15.099819 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5176f5-d11f-4b2b-bde3-db15566246f8 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:15.100329 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5176f5-d11f-4b2b-bde3-db15566246f8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:15.105131 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5176f5-d11f-4b2b-bde3-db15566246f8 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:15.106965 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 52/106] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:15 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:15.107983 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5176f5-d11f-4b2b-bde3-db15566246f8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:15.144280 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:15.217009 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5176f5-d11f-4b2b-bde3-db15566246f8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:15.239198 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:15.256511 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2618be-e343-481a-bc10-6eef406e3590 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:15.291915 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ebf05a-45a9-4d51-b550-fe0eaafd7301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:15.293131 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ebf05a-45a9-4d51-b550-fe0eaafd7301 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:15.293559 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ebf05a-45a9-4d51-b550-fe0eaafd7301 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:15.293978 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ebf05a-45a9-4d51-b550-fe0eaafd7301 None admin] PATH_INFO: `/v3/users/6f93dac3329a47ca90ee32eed0f164e7` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:15.295592 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da806388-3477-42f0-9b12-6068bafa80cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:15.296426 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da806388-3477-42f0-9b12-6068bafa80cd None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:15.296717 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da806388-3477-42f0-9b12-6068bafa80cd None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:15.297000 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da806388-3477-42f0-9b12-6068bafa80cd None admin] PATH_INFO: `/v3/users/aac9cdff3dc345eaa75a0bec09d65b4c` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:15.298819 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da806388-3477-42f0-9b12-6068bafa80cd None admin] RBAC: Authorizing `identity:get_user(user_id=aac9cdff3dc345eaa75a0bec09d65b4c)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:15.300298 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da806388-3477-42f0-9b12-6068bafa80cd None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:15.301962 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 53/107] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:00:15 2022] GET /identity/v3/users/aac9cdff3dc345eaa75a0bec09d65b4c => generated 275 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:15.303135 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9700a92-46e8-4373-877c-4b5a1e3c6632 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:15.303419 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9700a92-46e8-4373-877c-4b5a1e3c6632 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:15.310878 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9700a92-46e8-4373-877c-4b5a1e3c6632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:15.311012 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ebf05a-45a9-4d51-b550-fe0eaafd7301 None admin] RBAC: Authorizing `identity:get_user(user_id=6f93dac3329a47ca90ee32eed0f164e7)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:15.311012 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ebf05a-45a9-4d51-b550-fe0eaafd7301 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:15.312655 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 55/108] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:00:15 2022] GET /identity/v3/users/6f93dac3329a47ca90ee32eed0f164e7 => generated 289 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:00:15.318096 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ebf05a-45a9-4d51-b550-fe0eaafd7301 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:15.346576 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d9700a92-46e8-4373-877c-4b5a1e3c6632 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:15.348105 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d9700a92-46e8-4373-877c-4b5a1e3c6632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:15.419242 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:15.449240 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ebf05a-45a9-4d51-b550-fe0eaafd7301 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:15.493903 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 54/109] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:00:15.494788 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9700a92-46e8-4373-877c-4b5a1e3c6632 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:15.507597 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01215788-7312-4e87-b4ce-4d2cec9ed128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:15.508682 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01215788-7312-4e87-b4ce-4d2cec9ed128 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:15.509116 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01215788-7312-4e87-b4ce-4d2cec9ed128 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:15.509561 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01215788-7312-4e87-b4ce-4d2cec9ed128 None admin] PATH_INFO: `/v3/projects/00203c7cc02341ee8ee7a591095f0c7d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:15.519691 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01215788-7312-4e87-b4ce-4d2cec9ed128 None admin] RBAC: Authorizing `identity:get_project(project_id=00203c7cc02341ee8ee7a591095f0c7d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:15.521784 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01215788-7312-4e87-b4ce-4d2cec9ed128 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:15.523765 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 56/110] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:00:15 2022] GET /identity/v3/projects/00203c7cc02341ee8ee7a591095f0c7d => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:15.529599 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01215788-7312-4e87-b4ce-4d2cec9ed128 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:15.629109 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9700a92-46e8-4373-877c-4b5a1e3c6632 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:15.632189 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:15.663227 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01215788-7312-4e87-b4ce-4d2cec9ed128 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:15.668496 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ae1501-e9f5-4d56-aa37-3cf2ddefcff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:15.669163 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae1501-e9f5-4d56-aa37-3cf2ddefcff6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:15.669453 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae1501-e9f5-4d56-aa37-3cf2ddefcff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:15.669737 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae1501-e9f5-4d56-aa37-3cf2ddefcff6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:15.698771 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3b886d-3834-4d9f-bcaf-9df14436f0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:15.699789 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b886d-3834-4d9f-bcaf-9df14436f0aa None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:15.700234 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b886d-3834-4d9f-bcaf-9df14436f0aa None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:15.700504 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3b886d-3834-4d9f-bcaf-9df14436f0aa None admin] PATH_INFO: `/v3/projects/00203c7cc02341ee8ee7a591095f0c7d/users/aac9cdff3dc345eaa75a0bec09d65b4c/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:15.702404 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3b886d-3834-4d9f-bcaf-9df14436f0aa None admin] RBAC: Authorizing `identity:create_grant(project_id=00203c7cc02341ee8ee7a591095f0c7d, user_id=aac9cdff3dc345eaa75a0bec09d65b4c, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:15.703885 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3b886d-3834-4d9f-bcaf-9df14436f0aa None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:15.718806 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 57/111] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 15:00:15 2022] PUT /identity/v3/projects/00203c7cc02341ee8ee7a591095f0c7d/users/aac9cdff3dc345eaa75a0bec09d65b4c/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:00:15.719533 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3b886d-3834-4d9f-bcaf-9df14436f0aa None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:15.749453 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:15.775974 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ae1501-e9f5-4d56-aa37-3cf2ddefcff6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:15.777756 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ae1501-e9f5-4d56-aa37-3cf2ddefcff6 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:15.854601 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3b886d-3834-4d9f-bcaf-9df14436f0aa None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:15.894553 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df393b25-03f3-4839-b497-ad68cec867a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:15.895813 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df393b25-03f3-4839-b497-ad68cec867a6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:15.896257 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df393b25-03f3-4839-b497-ad68cec867a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:15.896678 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df393b25-03f3-4839-b497-ad68cec867a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:15.913090 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df393b25-03f3-4839-b497-ad68cec867a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:15.923635 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-df393b25-03f3-4839-b497-ad68cec867a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:00:15.923635 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:15.923635 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:15.923635 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:15.923635 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:15.923635 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:15.923635 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:15.923635 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:15.923635 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:15.923635 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:15.923635 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:15.923635 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:15.923635 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:15.923635 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:15.923635 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:15.923635 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:15.924562 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:15.924562 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:15.924562 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:15.924562 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:15.924562 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:15.924562 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:15.924562 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:15.924562 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:15.924562 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:15.924562 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:15.924562 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:15.924562 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:00:15.924562 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:15.924562 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:15.924562 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:15.924562 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:15.925575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:15.925575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:15.925575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:15.925575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:15.925575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:15.925575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:15.925575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:15.925575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:15.925575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:15.925575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:15.925575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:00:15.925575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:00:15.925575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 58/112] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 15:00:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:00:15.928159 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dce815-f4f0-406e-986d-137fb22f4264 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:15.928159 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dce815-f4f0-406e-986d-137fb22f4264 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:15.928322 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dce815-f4f0-406e-986d-137fb22f4264 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:15.930153 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 59/113] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:15.930872 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587ee9a-44e4-49a2-bd92-3520205b679e None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:15.931187 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587ee9a-44e4-49a2-bd92-3520205b679e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:15.931281 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 55/114] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:00:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 15:00:15.932655 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587ee9a-44e4-49a2-bd92-3520205b679e None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:15.932772 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ae1501-e9f5-4d56-aa37-3cf2ddefcff6 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:15.933117 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 60/115] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:15.936711 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:15.938479 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab1e8ac-03c4-4a94-b28c-feb4f60fef09 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:15.938804 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab1e8ac-03c4-4a94-b28c-feb4f60fef09 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:15.939138 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab1e8ac-03c4-4a94-b28c-feb4f60fef09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:15.959007 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1ab1e8ac-03c4-4a94-b28c-feb4f60fef09 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:15.960229 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1ab1e8ac-03c4-4a94-b28c-feb4f60fef09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:16.034184 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 61/116] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:00:16.035546 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcec64-53ee-48fb-9347-f09c91e370b3 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:16.035839 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcec64-53ee-48fb-9347-f09c91e370b3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:16.036122 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbcec64-53ee-48fb-9347-f09c91e370b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:16.043053 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:16.065772 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6bbcec64-53ee-48fb-9347-f09c91e370b3 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:16.067082 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6bbcec64-53ee-48fb-9347-f09c91e370b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:16.068848 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ae1501-e9f5-4d56-aa37-3cf2ddefcff6 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:16.116430 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb37a0-8112-46fb-9a12-16c097173fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:16.117108 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb37a0-8112-46fb-9a12-16c097173fa6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:16.117394 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb37a0-8112-46fb-9a12-16c097173fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:16.117674 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb37a0-8112-46fb-9a12-16c097173fa6 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:16.118425 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fb37a0-8112-46fb-9a12-16c097173fa6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:16.118732 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fb37a0-8112-46fb-9a12-16c097173fa6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:16.120145 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fb37a0-8112-46fb-9a12-16c097173fa6 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:16.129259 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 56/117] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:00:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:16.129876 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb37a0-8112-46fb-9a12-16c097173fa6 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:16.153095 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 62/118] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:00:16.153966 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbcec64-53ee-48fb-9347-f09c91e370b3 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:16.209011 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:16.244689 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb37a0-8112-46fb-9a12-16c097173fa6 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:16.264287 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbcec64-53ee-48fb-9347-f09c91e370b3 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:16.279589 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d815f-1a8d-4925-b8b2-7c7727ed9568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:16.280456 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d815f-1a8d-4925-b8b2-7c7727ed9568 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:16.280740 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d815f-1a8d-4925-b8b2-7c7727ed9568 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:16.281018 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d815f-1a8d-4925-b8b2-7c7727ed9568 None admin] PATH_INFO: `/v3/projects/00203c7cc02341ee8ee7a591095f0c7d` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:16.282250 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d815f-1a8d-4925-b8b2-7c7727ed9568 None admin] RBAC: Authorizing `identity:get_project(project_id=00203c7cc02341ee8ee7a591095f0c7d)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:16.284396 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d815f-1a8d-4925-b8b2-7c7727ed9568 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:16.286121 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 57/119] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:00:16 2022] GET /identity/v3/projects/00203c7cc02341ee8ee7a591095f0c7d => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:16.286912 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d815f-1a8d-4925-b8b2-7c7727ed9568 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:16.307740 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbb048b-d880-42e0-bb97-74be1f31a6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:16.308549 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb048b-d880-42e0-bb97-74be1f31a6a5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:16.308837 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb048b-d880-42e0-bb97-74be1f31a6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:16.309119 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbb048b-d880-42e0-bb97-74be1f31a6a5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:16.310300 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbb048b-d880-42e0-bb97-74be1f31a6a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:16.311773 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbb048b-d880-42e0-bb97-74be1f31a6a5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:16.314900 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 63/120] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:00:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:16.315317 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90365a4b-70ec-4bf7-960e-a1f2a7ba276a None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:16.315524 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90365a4b-70ec-4bf7-960e-a1f2a7ba276a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:16.315807 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90365a4b-70ec-4bf7-960e-a1f2a7ba276a None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:16.317044 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 64/121] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:16.317748 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90365a4b-70ec-4bf7-960e-a1f2a7ba276a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:16.322348 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:16.405228 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d815f-1a8d-4925-b8b2-7c7727ed9568 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:16.432894 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90365a4b-70ec-4bf7-960e-a1f2a7ba276a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:16.439850 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b752e-f514-4d67-9431-e70c9c43deae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:16.440735 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b752e-f514-4d67-9431-e70c9c43deae None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:16.441023 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b752e-f514-4d67-9431-e70c9c43deae None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:16.441305 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b752e-f514-4d67-9431-e70c9c43deae None admin] PATH_INFO: `/v3/users/9c29995c0b564cd7879c58c75dc099cb` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:16.442588 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b752e-f514-4d67-9431-e70c9c43deae None admin] RBAC: Authorizing `identity:get_user(user_id=9c29995c0b564cd7879c58c75dc099cb)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:16.444102 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3b752e-f514-4d67-9431-e70c9c43deae None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:16.445905 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 58/122] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:00:16 2022] GET /identity/v3/users/9c29995c0b564cd7879c58c75dc099cb => generated 247 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:16.446520 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b752e-f514-4d67-9431-e70c9c43deae None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:16.449121 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:16.476839 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef7dee-ccb0-494e-968e-697b544b12c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:16.477657 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef7dee-ccb0-494e-968e-697b544b12c2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:16.477945 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef7dee-ccb0-494e-968e-697b544b12c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:16.478341 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef7dee-ccb0-494e-968e-697b544b12c2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:16.489190 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ef7dee-ccb0-494e-968e-697b544b12c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:16.497441 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-97ef7dee-ccb0-494e-968e-697b544b12c2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 15:00:16.497441 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:16.497441 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:16.497441 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:16.497441 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:16.497441 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:16.497441 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:16.497441 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:16.497441 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:16.497441 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:16.497441 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:16.497441 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:16.497441 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:16.497441 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:16.497441 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:16.497441 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:16.498313 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:16.498313 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:16.498313 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:16.498313 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:16.498313 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:16.498313 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:16.498313 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:16.498313 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:16.498313 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:16.498313 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:16.498313 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:16.498313 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:00:16.498313 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:16.498313 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:16.498313 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:16.498313 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:16.499130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:16.499130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:16.499130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:16.499130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:16.499130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:16.499130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:16.499130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:16.499130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:16.499130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:16.499130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:16.499130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 15:00:16.499130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:00:16.499130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 65/123] 10.222.0.50 () {56 vars in 1114 bytes} [Fri May 27 15:00:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:00:16.499751 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef7dee-ccb0-494e-968e-697b544b12c2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:16.561731 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:16.565686 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b752e-f514-4d67-9431-e70c9c43deae None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:16.602041 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b173d9-aafb-489d-8292-3692eb86bb21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:16.602706 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b173d9-aafb-489d-8292-3692eb86bb21 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:16.603048 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b173d9-aafb-489d-8292-3692eb86bb21 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:16.603335 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b173d9-aafb-489d-8292-3692eb86bb21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:16.604169 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b173d9-aafb-489d-8292-3692eb86bb21 None admin] RBAC: Adding query filter params (role.id=1b313eb2327b44b7a11d92914db65bb9, scope.project.id=00203c7cc02341ee8ee7a591095f0c7d, user.id=6f93dac3329a47ca90ee32eed0f164e7) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:16.604488 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b173d9-aafb-489d-8292-3692eb86bb21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:16.605941 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b173d9-aafb-489d-8292-3692eb86bb21 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:16.606696 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef7dee-ccb0-494e-968e-697b544b12c2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:16.622885 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 59/124] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:00:16 2022] GET /identity/v3/role_assignments?user.id=6f93dac3329a47ca90ee32eed0f164e7&scope.project.id=00203c7cc02341ee8ee7a591095f0c7d&role.id=1b313eb2327b44b7a11d92914db65bb9 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:16.622885 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e835158d-5f6d-480c-96a4-d56d3c1fb112 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:16.622885 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e835158d-5f6d-480c-96a4-d56d3c1fb112 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:16.622885 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e835158d-5f6d-480c-96a4-d56d3c1fb112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:16.647846 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e835158d-5f6d-480c-96a4-d56d3c1fb112 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:16.650148 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e835158d-5f6d-480c-96a4-d56d3c1fb112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:16.655411 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4fcf36-9500-4b95-ae92-f10ca9b03414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:16.658873 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4fcf36-9500-4b95-ae92-f10ca9b03414 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:16.658873 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4fcf36-9500-4b95-ae92-f10ca9b03414 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:16.658873 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4fcf36-9500-4b95-ae92-f10ca9b03414 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:16.658873 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4fcf36-9500-4b95-ae92-f10ca9b03414 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:16.658873 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4fcf36-9500-4b95-ae92-f10ca9b03414 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:16.695765 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 66/125] 10.222.0.50 () {60 vars in 1141 bytes} [Fri May 27 15:00:16 2022] POST /identity/v3/projects => generated 302 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:00:16.697402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4fcf36-9500-4b95-ae92-f10ca9b03414 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:16.752236 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:16.768930 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 60/126] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:00:16.769976 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e835158d-5f6d-480c-96a4-d56d3c1fb112 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:16.824631 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4fcf36-9500-4b95-ae92-f10ca9b03414 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:16.844101 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:16.871353 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e2e23f-7c4a-404c-bd30-349dd5774e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:16.872423 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e2e23f-7c4a-404c-bd30-349dd5774e69 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:16.872817 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e2e23f-7c4a-404c-bd30-349dd5774e69 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:16.873201 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e2e23f-7c4a-404c-bd30-349dd5774e69 None admin] PATH_INFO: `/v3/projects/00203c7cc02341ee8ee7a591095f0c7d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:16.875239 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e2e23f-7c4a-404c-bd30-349dd5774e69 None admin] RBAC: Authorizing `identity:get_project(project_id=00203c7cc02341ee8ee7a591095f0c7d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:16.877743 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e2e23f-7c4a-404c-bd30-349dd5774e69 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:16.880115 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 67/127] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:00:16 2022] GET /identity/v3/projects/00203c7cc02341ee8ee7a591095f0c7d => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:16.884343 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e2e23f-7c4a-404c-bd30-349dd5774e69 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:16.969639 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e835158d-5f6d-480c-96a4-d56d3c1fb112 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:17.061716 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e2e23f-7c4a-404c-bd30-349dd5774e69 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:17.067544 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e28e63-14ba-410a-8004-e4410e4cdd53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:17.068237 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e28e63-14ba-410a-8004-e4410e4cdd53 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:17.068525 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e28e63-14ba-410a-8004-e4410e4cdd53 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:17.068807 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e28e63-14ba-410a-8004-e4410e4cdd53 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:17.069570 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e28e63-14ba-410a-8004-e4410e4cdd53 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:17.069882 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e28e63-14ba-410a-8004-e4410e4cdd53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:17.071319 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e28e63-14ba-410a-8004-e4410e4cdd53 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:17.083920 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 61/128] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:00:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:17.085263 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e28e63-14ba-410a-8004-e4410e4cdd53 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:17.115719 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e46728-cb17-4355-b6ec-b09cee91f620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:17.117071 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e46728-cb17-4355-b6ec-b09cee91f620 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:17.117435 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e46728-cb17-4355-b6ec-b09cee91f620 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:17.117792 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e46728-cb17-4355-b6ec-b09cee91f620 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:17.133775 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e46728-cb17-4355-b6ec-b09cee91f620 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:17.143899 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-93e46728-cb17-4355-b6ec-b09cee91f620 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 15:00:17.143899 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:17.143899 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:17.143899 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:17.143899 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:17.143899 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:17.143899 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:17.143899 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:17.143899 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:17.143899 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:17.143899 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:17.143899 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:17.143899 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:17.143899 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:17.143899 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:17.143899 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:17.144920 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:17.144920 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:17.144920 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:17.144920 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:17.144920 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:17.144920 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:17.144920 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:17.144920 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:17.144920 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:17.144920 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:17.144920 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:17.144920 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:00:17.144920 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:17.144920 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:17.144920 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:17.144920 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:17.145726 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:17.145726 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:17.145726 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:17.145726 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:17.145726 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:17.145726 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:17.145726 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:17.145726 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:17.145726 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:17.145726 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:17.145726 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 15:00:17.145726 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:00:17.145726 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 68/129] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:00:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:00:17.146395 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e46728-cb17-4355-b6ec-b09cee91f620 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:17.173709 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:17.201086 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:17.226244 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e28e63-14ba-410a-8004-e4410e4cdd53 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:17.264309 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deea09c-bb25-4461-929d-20504ed9363e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:17.265726 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deea09c-bb25-4461-929d-20504ed9363e None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:17.266341 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deea09c-bb25-4461-929d-20504ed9363e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:17.266832 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7deea09c-bb25-4461-929d-20504ed9363e None admin] PATH_INFO: `/v3/projects/00203c7cc02341ee8ee7a591095f0c7d/users/9c29995c0b564cd7879c58c75dc099cb/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:17.269333 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deea09c-bb25-4461-929d-20504ed9363e None admin] RBAC: Authorizing `identity:create_grant(project_id=00203c7cc02341ee8ee7a591095f0c7d, user_id=9c29995c0b564cd7879c58c75dc099cb, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:17.271070 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7deea09c-bb25-4461-929d-20504ed9363e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:17.289761 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 62/130] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 15:00:17 2022] PUT /identity/v3/projects/00203c7cc02341ee8ee7a591095f0c7d/users/9c29995c0b564cd7879c58c75dc099cb/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:00:17.291114 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deea09c-bb25-4461-929d-20504ed9363e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:17.344649 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e46728-cb17-4355-b6ec-b09cee91f620 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:17.388931 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4545e2d-04cc-4079-aafb-936f2101fdc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:17.390240 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4545e2d-04cc-4079-aafb-936f2101fdc9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:17.390240 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4545e2d-04cc-4079-aafb-936f2101fdc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:17.390491 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4545e2d-04cc-4079-aafb-936f2101fdc9 None admin] PATH_INFO: `/v3/users/aac9cdff3dc345eaa75a0bec09d65b4c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:17.391870 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4545e2d-04cc-4079-aafb-936f2101fdc9 None admin] RBAC: Authorizing `identity:get_user(user_id=aac9cdff3dc345eaa75a0bec09d65b4c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:17.393473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4545e2d-04cc-4079-aafb-936f2101fdc9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:17.398992 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 69/131] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:00:17 2022] GET /identity/v3/users/aac9cdff3dc345eaa75a0bec09d65b4c => generated 275 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:17.401191 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4545e2d-04cc-4079-aafb-936f2101fdc9 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:17.500548 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7deea09c-bb25-4461-929d-20504ed9363e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:17.509285 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4545e2d-04cc-4079-aafb-936f2101fdc9 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:17.527368 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:17.541599 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:17.547345 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff5644-45b1-454d-a740-6190ab1755a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:17.548362 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff5644-45b1-454d-a740-6190ab1755a0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:17.548746 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff5644-45b1-454d-a740-6190ab1755a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:17.549105 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff5644-45b1-454d-a740-6190ab1755a0 None admin] PATH_INFO: `/v3/projects/00203c7cc02341ee8ee7a591095f0c7d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:17.550686 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff5644-45b1-454d-a740-6190ab1755a0 None admin] RBAC: Authorizing `identity:get_project(project_id=00203c7cc02341ee8ee7a591095f0c7d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:17.552316 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff5644-45b1-454d-a740-6190ab1755a0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:17.554307 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 70/132] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:00:17 2022] GET /identity/v3/projects/00203c7cc02341ee8ee7a591095f0c7d => generated 298 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:00:17.560431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff5644-45b1-454d-a740-6190ab1755a0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:17.584925 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe31f34-9817-41f7-a880-5554e4a341da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:17.586303 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe31f34-9817-41f7-a880-5554e4a341da None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:17.586768 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe31f34-9817-41f7-a880-5554e4a341da None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:17.587243 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe31f34-9817-41f7-a880-5554e4a341da None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:17.588456 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe31f34-9817-41f7-a880-5554e4a341da None admin] RBAC: Adding query filter params (name=member) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:17.588956 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe31f34-9817-41f7-a880-5554e4a341da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:17.590533 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe31f34-9817-41f7-a880-5554e4a341da None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:17.607977 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 63/133] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 15:00:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:17.614060 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe31f34-9817-41f7-a880-5554e4a341da None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:17.696690 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff5644-45b1-454d-a740-6190ab1755a0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:17.746071 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad511c7c-8d8f-43ac-bf89-6836390b7bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:17.747563 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad511c7c-8d8f-43ac-bf89-6836390b7bf8 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:17.747926 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad511c7c-8d8f-43ac-bf89-6836390b7bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:17.748280 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad511c7c-8d8f-43ac-bf89-6836390b7bf8 None admin] PATH_INFO: `/v3/projects/00203c7cc02341ee8ee7a591095f0c7d/users/aac9cdff3dc345eaa75a0bec09d65b4c/roles/ad9498a5e2064a468acc151fb2bce49e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:17.752419 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad511c7c-8d8f-43ac-bf89-6836390b7bf8 None admin] RBAC: Authorizing `identity:create_grant(project_id=00203c7cc02341ee8ee7a591095f0c7d, user_id=aac9cdff3dc345eaa75a0bec09d65b4c, role_id=ad9498a5e2064a468acc151fb2bce49e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:17.754181 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad511c7c-8d8f-43ac-bf89-6836390b7bf8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:17.769377 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe31f34-9817-41f7-a880-5554e4a341da None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:17.781686 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 71/134] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 15:00:17 2022] PUT /identity/v3/projects/00203c7cc02341ee8ee7a591095f0c7d/users/aac9cdff3dc345eaa75a0bec09d65b4c/roles/ad9498a5e2064a468acc151fb2bce49e => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:00:17.844462 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1b7cb-42f3-46d8-8e72-962891a27744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:17.845463 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1b7cb-42f3-46d8-8e72-962891a27744 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:17.845789 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1b7cb-42f3-46d8-8e72-962891a27744 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:17.846096 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf1b7cb-42f3-46d8-8e72-962891a27744 None admin] PATH_INFO: `/v3/users/aac9cdff3dc345eaa75a0bec09d65b4c` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:17.851894 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf1b7cb-42f3-46d8-8e72-962891a27744 None admin] RBAC: Authorizing `identity:get_user(user_id=aac9cdff3dc345eaa75a0bec09d65b4c)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:17.853437 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf1b7cb-42f3-46d8-8e72-962891a27744 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:17.855683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 64/135] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:00:17 2022] GET /identity/v3/users/aac9cdff3dc345eaa75a0bec09d65b4c => generated 275 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:00:17.861592 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad511c7c-8d8f-43ac-bf89-6836390b7bf8 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:17.936214 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:18.042958 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad511c7c-8d8f-43ac-bf89-6836390b7bf8 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:18.097178 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c9903-ab81-492e-8923-abf9634d31fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:18.098097 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c9903-ab81-492e-8923-abf9634d31fe None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:18.098388 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c9903-ab81-492e-8923-abf9634d31fe None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:18.098666 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c9903-ab81-492e-8923-abf9634d31fe None admin] PATH_INFO: `/v3/projects/a8102aa3c6634ff6b42c5df957f24182` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:18.100061 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9c9903-ab81-492e-8923-abf9634d31fe None admin] RBAC: Authorizing `identity:get_project(project_id=a8102aa3c6634ff6b42c5df957f24182)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:18.101602 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9c9903-ab81-492e-8923-abf9634d31fe None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:18.103695 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 72/136] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:00:17 2022] GET /identity/v3/projects/a8102aa3c6634ff6b42c5df957f24182 => generated 312 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:18.109704 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1b7cb-42f3-46d8-8e72-962891a27744 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:18.177878 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:18.240210 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf1b7cb-42f3-46d8-8e72-962891a27744 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:18.289083 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2992678-6d26-46a0-bf20-11a1a5219c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:18.290146 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2992678-6d26-46a0-bf20-11a1a5219c76 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:18.290525 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2992678-6d26-46a0-bf20-11a1a5219c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:18.290941 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2992678-6d26-46a0-bf20-11a1a5219c76 None admin] PATH_INFO: `/v3/projects/a8102aa3c6634ff6b42c5df957f24182/users/aac9cdff3dc345eaa75a0bec09d65b4c/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:18.293516 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2992678-6d26-46a0-bf20-11a1a5219c76 None admin] RBAC: Authorizing `identity:create_grant(project_id=a8102aa3c6634ff6b42c5df957f24182, user_id=aac9cdff3dc345eaa75a0bec09d65b4c, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:18.295070 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2992678-6d26-46a0-bf20-11a1a5219c76 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:18.321391 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 65/137] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 15:00:18 2022] PUT /identity/v3/projects/a8102aa3c6634ff6b42c5df957f24182/users/aac9cdff3dc345eaa75a0bec09d65b4c/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:00:18.428066 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d170a13-3472-4186-bd48-bcfb0a3517b1 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:18.428361 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d170a13-3472-4186-bd48-bcfb0a3517b1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:18.428649 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d170a13-3472-4186-bd48-bcfb0a3517b1 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:18.430041 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 73/138] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:18.435656 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feaf92a-e384-4b8e-abd0-20e91898d715 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:18.435958 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feaf92a-e384-4b8e-abd0-20e91898d715 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:18.436245 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feaf92a-e384-4b8e-abd0-20e91898d715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:18.454048 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-9feaf92a-e384-4b8e-abd0-20e91898d715 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:18.455324 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9feaf92a-e384-4b8e-abd0-20e91898d715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:18.541145 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 66/139] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:00:18.565560 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d170a13-3472-4186-bd48-bcfb0a3517b1 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:18.675902 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d170a13-3472-4186-bd48-bcfb0a3517b1 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:18.693930 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:18.713762 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622834cf-2ae1-4a23-8ac6-8486872f4c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:18.714568 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622834cf-2ae1-4a23-8ac6-8486872f4c50 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:18.714873 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622834cf-2ae1-4a23-8ac6-8486872f4c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:18.715162 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622834cf-2ae1-4a23-8ac6-8486872f4c50 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:18.726388 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622834cf-2ae1-4a23-8ac6-8486872f4c50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:18.734315 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-622834cf-2ae1-4a23-8ac6-8486872f4c50 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 15:00:18.734315 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:18.734315 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:18.734315 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:18.734315 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:18.734315 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:18.734315 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:18.734315 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:18.734315 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:18.734315 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:18.734315 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:18.734315 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:18.734315 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:18.734315 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:18.734315 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:18.734315 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:18.735190 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:18.735190 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:18.735190 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:18.735190 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:18.735190 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:18.735190 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:18.735190 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:18.735190 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:18.735190 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:18.735190 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:18.735190 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:18.735190 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:00:18.735190 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:18.735190 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:18.735190 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:18.735190 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:18.737342 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:18.737342 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:18.737342 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:18.737342 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:18.737342 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:18.737342 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:18.737342 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:18.737342 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:18.737342 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:18.737342 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:18.737342 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 15:00:18.737342 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:00:18.737342 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 74/140] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:00:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:00:18.742031 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feaf92a-e384-4b8e-abd0-20e91898d715 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:18.809479 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:18.892554 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feaf92a-e384-4b8e-abd0-20e91898d715 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:18.939717 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62eefa6-e4b1-4309-80ef-c57c1315c149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:18.940799 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62eefa6-e4b1-4309-80ef-c57c1315c149 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:18.941339 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62eefa6-e4b1-4309-80ef-c57c1315c149 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:18.941821 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62eefa6-e4b1-4309-80ef-c57c1315c149 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:18.942780 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62eefa6-e4b1-4309-80ef-c57c1315c149 None admin] RBAC: Adding query filter params (name=member) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:18.943307 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62eefa6-e4b1-4309-80ef-c57c1315c149 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:18.944911 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62eefa6-e4b1-4309-80ef-c57c1315c149 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:18.958957 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 67/141] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 15:00:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:18.964881 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622834cf-2ae1-4a23-8ac6-8486872f4c50 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:19.091374 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622834cf-2ae1-4a23-8ac6-8486872f4c50 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:19.130361 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e2baa2-74ce-4fb3-9ea5-7f02984e3c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:19.131340 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e2baa2-74ce-4fb3-9ea5-7f02984e3c2e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:19.131635 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e2baa2-74ce-4fb3-9ea5-7f02984e3c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:19.131960 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e2baa2-74ce-4fb3-9ea5-7f02984e3c2e None admin] PATH_INFO: `/v3/users/aac9cdff3dc345eaa75a0bec09d65b4c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:19.133198 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e2baa2-74ce-4fb3-9ea5-7f02984e3c2e None admin] RBAC: Authorizing `identity:get_user(user_id=aac9cdff3dc345eaa75a0bec09d65b4c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:19.134654 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e2baa2-74ce-4fb3-9ea5-7f02984e3c2e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:19.136655 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 75/142] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:00:18 2022] GET /identity/v3/users/aac9cdff3dc345eaa75a0bec09d65b4c => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:19.142733 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62eefa6-e4b1-4309-80ef-c57c1315c149 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:19.287963 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:19.291661 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62eefa6-e4b1-4309-80ef-c57c1315c149 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:19.327289 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deec866b-9591-479f-a015-dffdab057a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:19.327948 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec866b-9591-479f-a015-dffdab057a8a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:19.328238 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec866b-9591-479f-a015-dffdab057a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:19.328522 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec866b-9591-479f-a015-dffdab057a8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:19.410956 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deec866b-9591-479f-a015-dffdab057a8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:19.412372 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deec866b-9591-479f-a015-dffdab057a8a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:19.524695 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:19.528583 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 68/143] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:00:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 15:00:19.534616 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e2baa2-74ce-4fb3-9ea5-7f02984e3c2e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:19.570157 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:19.639400 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e2baa2-74ce-4fb3-9ea5-7f02984e3c2e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:19.675157 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2735e5c-8d06-4fdc-9f9f-161ebf86bd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:19.675950 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2735e5c-8d06-4fdc-9f9f-161ebf86bd31 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:19.676239 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2735e5c-8d06-4fdc-9f9f-161ebf86bd31 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:19.676519 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2735e5c-8d06-4fdc-9f9f-161ebf86bd31 None admin] PATH_INFO: `/v3/projects/00203c7cc02341ee8ee7a591095f0c7d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:19.677719 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2735e5c-8d06-4fdc-9f9f-161ebf86bd31 None admin] RBAC: Authorizing `identity:get_project(project_id=00203c7cc02341ee8ee7a591095f0c7d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:19.679271 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2735e5c-8d06-4fdc-9f9f-161ebf86bd31 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:19.681136 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 76/144] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:00:19 2022] GET /identity/v3/projects/00203c7cc02341ee8ee7a591095f0c7d => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:19.687010 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deec866b-9591-479f-a015-dffdab057a8a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:19.790125 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deec866b-9591-479f-a015-dffdab057a8a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:19.824195 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069aa038-34d7-4b8a-b1e3-68fada96dfa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:19.824857 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069aa038-34d7-4b8a-b1e3-68fada96dfa1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:19.825140 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069aa038-34d7-4b8a-b1e3-68fada96dfa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:19.825415 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069aa038-34d7-4b8a-b1e3-68fada96dfa1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:19.826309 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069aa038-34d7-4b8a-b1e3-68fada96dfa1 None admin] RBAC: Adding query filter params (role.id=8442c42e847a4b38b76f3a7fbe9ada85, scope.project.id=00203c7cc02341ee8ee7a591095f0c7d, user.id=aac9cdff3dc345eaa75a0bec09d65b4c) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:19.826629 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069aa038-34d7-4b8a-b1e3-68fada96dfa1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:19.828108 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069aa038-34d7-4b8a-b1e3-68fada96dfa1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:19.835339 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 69/145] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:00:19 2022] GET /identity/v3/role_assignments?user.id=aac9cdff3dc345eaa75a0bec09d65b4c&scope.project.id=00203c7cc02341ee8ee7a591095f0c7d&role.id=8442c42e847a4b38b76f3a7fbe9ada85 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:20.250677 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847e6c8-4d47-48cd-9023-0d097efd7b01 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:20.251592 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847e6c8-4d47-48cd-9023-0d097efd7b01 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:20.252189 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847e6c8-4d47-48cd-9023-0d097efd7b01 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:20.253777 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 77/146] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:20.259121 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b44e60-ed52-4b6f-ac6b-f6cfedeff7dc None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:20.259665 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b44e60-ed52-4b6f-ac6b-f6cfedeff7dc None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:20.260149 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b44e60-ed52-4b6f-ac6b-f6cfedeff7dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:20.281555 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-75b44e60-ed52-4b6f-ac6b-f6cfedeff7dc None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:20.283802 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-75b44e60-ed52-4b6f-ac6b-f6cfedeff7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:20.353645 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:20.364318 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 70/147] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:00:20.387433 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c847e6c8-4d47-48cd-9023-0d097efd7b01 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:20.515796 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c847e6c8-4d47-48cd-9023-0d097efd7b01 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:20.518166 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:20.555253 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a11b5b-a5be-4892-ba26-d4f93dc883b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:20.556057 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a11b5b-a5be-4892-ba26-d4f93dc883b2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:20.556341 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a11b5b-a5be-4892-ba26-d4f93dc883b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:20.556614 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a11b5b-a5be-4892-ba26-d4f93dc883b2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:20.567828 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a11b5b-a5be-4892-ba26-d4f93dc883b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:20.575789 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-31a11b5b-a5be-4892-ba26-d4f93dc883b2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 15:00:20.575789 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:20.575789 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:20.575789 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:20.575789 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:20.575789 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:20.575789 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:20.575789 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:20.575789 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:20.575789 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:20.575789 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:20.575789 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:20.575789 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:20.575789 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:20.575789 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:20.575789 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:20.577091 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:20.577091 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:20.577091 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:20.577091 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:20.577091 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:20.577091 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:20.577091 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:20.577091 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:20.577091 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:20.577091 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:20.577091 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:20.577091 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:00:20.577091 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:20.577091 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:20.577091 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:20.577091 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:20.577863 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:20.577863 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:20.577863 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:20.577863 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:20.577863 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:20.577863 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:20.577863 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:20.577863 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:20.577863 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:20.577863 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:20.577863 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 15:00:20.577863 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:00:20.577863 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 78/148] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:00:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:00:20.583401 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b44e60-ed52-4b6f-ac6b-f6cfedeff7dc None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:20.597510 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f8fe0c-7f19-4848-a6f9-5d2d5a087fd9 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:20.597809 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f8fe0c-7f19-4848-a6f9-5d2d5a087fd9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:20.598073 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f8fe0c-7f19-4848-a6f9-5d2d5a087fd9 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:20.603081 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 79/149] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:20.608394 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a8fd5-057d-48e6-8f47-bec1cb1cd5bf None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:20.608681 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a8fd5-057d-48e6-8f47-bec1cb1cd5bf None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:20.608963 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9a8fd5-057d-48e6-8f47-bec1cb1cd5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:20.635141 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8f9a8fd5-057d-48e6-8f47-bec1cb1cd5bf None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:20.636345 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8f9a8fd5-057d-48e6-8f47-bec1cb1cd5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:20.768429 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b44e60-ed52-4b6f-ac6b-f6cfedeff7dc None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:20.772010 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 80/150] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 15:00:20.796353 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9a8fd5-057d-48e6-8f47-bec1cb1cd5bf None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:20.834555 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311a013e-3217-4efc-b6f6-f5e89637825f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:20.835333 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a013e-3217-4efc-b6f6-f5e89637825f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:20.835684 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a013e-3217-4efc-b6f6-f5e89637825f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:20.836419 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311a013e-3217-4efc-b6f6-f5e89637825f None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:20.836951 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311a013e-3217-4efc-b6f6-f5e89637825f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:20.837263 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311a013e-3217-4efc-b6f6-f5e89637825f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:20.838970 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311a013e-3217-4efc-b6f6-f5e89637825f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:20.860311 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 71/151] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 15:00:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:20.870772 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311a013e-3217-4efc-b6f6-f5e89637825f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:20.967779 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:20.983521 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:21.001921 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9a8fd5-057d-48e6-8f47-bec1cb1cd5bf None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:21.049425 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311a013e-3217-4efc-b6f6-f5e89637825f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:21.063101 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eafcda-2f2f-4da3-b279-7ae4ee412a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:21.064491 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eafcda-2f2f-4da3-b279-7ae4ee412a9a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:21.064846 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eafcda-2f2f-4da3-b279-7ae4ee412a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:21.065154 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eafcda-2f2f-4da3-b279-7ae4ee412a9a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:21.066579 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eafcda-2f2f-4da3-b279-7ae4ee412a9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:21.068350 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eafcda-2f2f-4da3-b279-7ae4ee412a9a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:21.070785 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 81/152] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:00:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:00:21.072763 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b6f8d-c2f9-44f5-96c6-d86e77f7619e None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:21.073065 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b6f8d-c2f9-44f5-96c6-d86e77f7619e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:21.073353 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571b6f8d-c2f9-44f5-96c6-d86e77f7619e None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:21.074599 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 82/153] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:21.080229 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571b6f8d-c2f9-44f5-96c6-d86e77f7619e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:21.095820 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f5432e-4561-451f-8d27-a8c5e46ca34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:21.096707 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5432e-4561-451f-8d27-a8c5e46ca34a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:21.096991 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5432e-4561-451f-8d27-a8c5e46ca34a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:21.097268 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f5432e-4561-451f-8d27-a8c5e46ca34a None admin] PATH_INFO: `/v3/users/6f93dac3329a47ca90ee32eed0f164e7` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:21.098484 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f5432e-4561-451f-8d27-a8c5e46ca34a None admin] RBAC: Authorizing `identity:get_user(user_id=6f93dac3329a47ca90ee32eed0f164e7)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:21.100012 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f5432e-4561-451f-8d27-a8c5e46ca34a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:21.101937 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 72/154] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:00:20 2022] GET /identity/v3/users/6f93dac3329a47ca90ee32eed0f164e7 => generated 289 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:21.107358 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48820e78-cfd0-467f-a2f7-99f711ebe98a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:21.107659 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48820e78-cfd0-467f-a2f7-99f711ebe98a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:21.107942 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48820e78-cfd0-467f-a2f7-99f711ebe98a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:21.124452 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-48820e78-cfd0-467f-a2f7-99f711ebe98a None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:21.125631 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-48820e78-cfd0-467f-a2f7-99f711ebe98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:21.202260 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 73/155] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:00:21.206950 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571b6f8d-c2f9-44f5-96c6-d86e77f7619e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:21.207053 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e142e1-826c-4623-8144-10f549147625 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:21.207053 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e142e1-826c-4623-8144-10f549147625 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:21.207053 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e142e1-826c-4623-8144-10f549147625 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:21.207053 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 74/156] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:21.207053 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e142e1-826c-4623-8144-10f549147625 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:21.245712 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c49ced3-e380-47f1-a570-3b23de122d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:21.246403 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c49ced3-e380-47f1-a570-3b23de122d86 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:21.246682 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c49ced3-e380-47f1-a570-3b23de122d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:21.246990 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c49ced3-e380-47f1-a570-3b23de122d86 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:21.247911 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c49ced3-e380-47f1-a570-3b23de122d86 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:21.249203 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c49ced3-e380-47f1-a570-3b23de122d86 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:21.272485 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 83/157] 10.222.0.50 () {60 vars in 1132 bytes} [Fri May 27 15:00:21 2022] POST /identity/v3/users => generated 283 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:00:21.274569 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057c36e-419d-4b7b-8e0f-c773d782b3a1 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:21.275207 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057c36e-419d-4b7b-8e0f-c773d782b3a1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:21.275774 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057c36e-419d-4b7b-8e0f-c773d782b3a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:21.287189 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:21.301182 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3057c36e-419d-4b7b-8e0f-c773d782b3a1 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:21.302320 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3057c36e-419d-4b7b-8e0f-c773d782b3a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:21.316846 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:21.348109 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e142e1-826c-4623-8144-10f549147625 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:21.383600 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 84/158] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:00:21.384528 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3057c36e-419d-4b7b-8e0f-c773d782b3a1 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:21.391114 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef05a3d2-521e-4570-b1a6-3e3c043ae171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:21.392622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05a3d2-521e-4570-b1a6-3e3c043ae171 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:21.393156 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05a3d2-521e-4570-b1a6-3e3c043ae171 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:21.393626 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05a3d2-521e-4570-b1a6-3e3c043ae171 None admin] PATH_INFO: `/v3/projects/00203c7cc02341ee8ee7a591095f0c7d` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:21.395117 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef05a3d2-521e-4570-b1a6-3e3c043ae171 None admin] RBAC: Authorizing `identity:get_project(project_id=00203c7cc02341ee8ee7a591095f0c7d)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:21.396828 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef05a3d2-521e-4570-b1a6-3e3c043ae171 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:21.398750 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 75/159] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:00:21 2022] GET /identity/v3/projects/00203c7cc02341ee8ee7a591095f0c7d => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:21.404762 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef05a3d2-521e-4570-b1a6-3e3c043ae171 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:21.512261 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3057c36e-419d-4b7b-8e0f-c773d782b3a1 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:21.531746 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef05a3d2-521e-4570-b1a6-3e3c043ae171 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:21.556066 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c439e9d-4088-407d-a093-0fc101f558b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:21.556812 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c439e9d-4088-407d-a093-0fc101f558b5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:21.557098 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c439e9d-4088-407d-a093-0fc101f558b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:21.557374 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c439e9d-4088-407d-a093-0fc101f558b5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:21.562890 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:21.572672 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c439e9d-4088-407d-a093-0fc101f558b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:21.575832 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1390e65-608b-4efc-b288-f1d0f4b23f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:21.576716 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1390e65-608b-4efc-b288-f1d0f4b23f8d None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:21.577000 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1390e65-608b-4efc-b288-f1d0f4b23f8d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:21.577276 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1390e65-608b-4efc-b288-f1d0f4b23f8d None admin] PATH_INFO: `/v3/projects/00203c7cc02341ee8ee7a591095f0c7d/users/6f93dac3329a47ca90ee32eed0f164e7/roles/1b313eb2327b44b7a11d92914db65bb9` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:21.580230 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1390e65-608b-4efc-b288-f1d0f4b23f8d None admin] RBAC: Authorizing `identity:create_grant(project_id=00203c7cc02341ee8ee7a591095f0c7d, user_id=6f93dac3329a47ca90ee32eed0f164e7, role_id=1b313eb2327b44b7a11d92914db65bb9)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:21.581684 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1390e65-608b-4efc-b288-f1d0f4b23f8d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:21.581789 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-2c439e9d-4088-407d-a093-0fc101f558b5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 15:00:21.581789 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:21.581789 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:21.581789 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:21.581789 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:21.581789 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:21.581789 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:21.581789 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:21.581789 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:21.581789 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:21.581789 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:21.581789 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:21.581789 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:21.581789 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:21.581789 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:21.581789 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:21.582971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:21.582971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:21.582971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:21.582971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:21.582971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:21.582971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:21.582971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:21.582971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:21.582971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:21.582971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:21.582971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:21.582971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:00:21.582971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:21.582971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:21.582971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:21.582971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:21.584124 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:21.584124 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:21.584124 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:21.584124 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:21.584124 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:21.584124 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:21.584124 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:21.584124 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:21.584124 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:21.584124 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:21.584124 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 15:00:21.584124 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:00:21.584124 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 85/160] 10.222.0.50 () {56 vars in 1114 bytes} [Fri May 27 15:00:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:00:21.584124 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c439e9d-4088-407d-a093-0fc101f558b5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:21.594212 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:21.604570 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 76/161] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 15:00:21 2022] PUT /identity/v3/projects/00203c7cc02341ee8ee7a591095f0c7d/users/6f93dac3329a47ca90ee32eed0f164e7/roles/1b313eb2327b44b7a11d92914db65bb9 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:00:21.606496 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3782549-81ab-422b-b773-a595da41e965 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:21.607329 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3782549-81ab-422b-b773-a595da41e965 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:21.607662 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3782549-81ab-422b-b773-a595da41e965 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:21.609201 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 77/162] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:21.611127 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3782549-81ab-422b-b773-a595da41e965 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:21.737496 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c439e9d-4088-407d-a093-0fc101f558b5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:21.770396 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3782549-81ab-422b-b773-a595da41e965 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:21.793746 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bdf1ec-9b8b-474f-86e3-1940e5423f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:21.794954 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bdf1ec-9b8b-474f-86e3-1940e5423f27 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:21.795462 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bdf1ec-9b8b-474f-86e3-1940e5423f27 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:21.795957 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bdf1ec-9b8b-474f-86e3-1940e5423f27 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:21.810108 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bdf1ec-9b8b-474f-86e3-1940e5423f27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:21.818476 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-d4bdf1ec-9b8b-474f-86e3-1940e5423f27 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:00:21.818476 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:21.818476 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:21.818476 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:21.818476 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:21.818476 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:21.818476 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:21.818476 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:21.818476 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:21.818476 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:21.818476 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:21.818476 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:21.818476 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:21.818476 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:21.818476 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:21.818476 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:21.819519 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:21.819519 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:21.819519 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:21.819519 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:21.819519 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:21.819519 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:21.819519 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:21.819519 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:21.819519 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:21.819519 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:21.819519 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:21.819519 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:00:21.819519 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:21.819519 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:21.819519 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:21.819519 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:21.820324 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:21.820324 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:21.820324 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:21.820324 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:21.820324 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:21.820324 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:21.820324 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:21.820324 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:21.820324 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:21.820324 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:21.820324 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:00:21.820324 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:00:21.821419 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7682fa07-b276-4f67-8154-53d242c4c608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:21.821853 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682fa07-b276-4f67-8154-53d242c4c608 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:21.822137 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682fa07-b276-4f67-8154-53d242c4c608 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:21.822412 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682fa07-b276-4f67-8154-53d242c4c608 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:21.823220 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7682fa07-b276-4f67-8154-53d242c4c608 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:21.823528 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7682fa07-b276-4f67-8154-53d242c4c608 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:21.824057 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 86/163] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 15:00:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:00:21.824990 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7682fa07-b276-4f67-8154-53d242c4c608 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:21.825794 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b321e5-2251-4292-8056-da691aa77205 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:21.826892 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b321e5-2251-4292-8056-da691aa77205 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:21.827388 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b321e5-2251-4292-8056-da691aa77205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:21.840542 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 78/164] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:00:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:21.841490 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7682fa07-b276-4f67-8154-53d242c4c608 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:21.850332 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-43b321e5-2251-4292-8056-da691aa77205 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:21.851916 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-43b321e5-2251-4292-8056-da691aa77205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:21.899166 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:21.929895 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:21.941609 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 87/165] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:00:21.942289 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b321e5-2251-4292-8056-da691aa77205 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:21.985372 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7682fa07-b276-4f67-8154-53d242c4c608 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:22.033803 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7850a15b-037d-438d-bbd6-ee609be703ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:22.034815 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7850a15b-037d-438d-bbd6-ee609be703ad None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:22.035353 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7850a15b-037d-438d-bbd6-ee609be703ad None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:22.035878 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7850a15b-037d-438d-bbd6-ee609be703ad None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:22.036873 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7850a15b-037d-438d-bbd6-ee609be703ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:22.037405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7850a15b-037d-438d-bbd6-ee609be703ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:22.039010 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7850a15b-037d-438d-bbd6-ee609be703ad None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:22.049347 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 79/166] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:00:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:22.050494 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7850a15b-037d-438d-bbd6-ee609be703ad None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:22.092517 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b321e5-2251-4292-8056-da691aa77205 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:22.129369 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88ff680-91ef-4078-b6ee-2541013a6d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:22.130251 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ff680-91ef-4078-b6ee-2541013a6d13 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:22.130534 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ff680-91ef-4078-b6ee-2541013a6d13 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:22.130808 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88ff680-91ef-4078-b6ee-2541013a6d13 None admin] PATH_INFO: `/v3/users/aac9cdff3dc345eaa75a0bec09d65b4c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:22.132058 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88ff680-91ef-4078-b6ee-2541013a6d13 None admin] RBAC: Authorizing `identity:get_user(user_id=aac9cdff3dc345eaa75a0bec09d65b4c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:22.133841 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88ff680-91ef-4078-b6ee-2541013a6d13 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:22.135335 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 88/167] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:00:21 2022] GET /identity/v3/users/aac9cdff3dc345eaa75a0bec09d65b4c => generated 275 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:22.135951 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88ff680-91ef-4078-b6ee-2541013a6d13 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:22.170269 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7850a15b-037d-438d-bbd6-ee609be703ad None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:22.183247 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:22.204879 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:22.222677 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a40e87e-5581-482d-9535-806383ef0b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:22.223607 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40e87e-5581-482d-9535-806383ef0b9d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:22.223898 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40e87e-5581-482d-9535-806383ef0b9d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:22.224175 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a40e87e-5581-482d-9535-806383ef0b9d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:22.240585 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a40e87e-5581-482d-9535-806383ef0b9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:22.245791 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88ff680-91ef-4078-b6ee-2541013a6d13 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:22.261066 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-3a40e87e-5581-482d-9535-806383ef0b9d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 15:00:22.261066 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:22.261066 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:22.261066 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:22.261066 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:22.261066 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:22.261066 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:22.261066 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:22.261066 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:22.261066 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:22.261066 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:22.261066 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:22.261066 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:22.261066 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:22.261066 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:22.261066 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:22.262028 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:22.262028 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:22.262028 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:22.262028 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:22.262028 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:22.262028 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:22.262028 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:22.262028 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:22.262028 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:22.262028 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:22.262028 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:22.262028 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:00:22.262028 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:22.262028 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:22.262028 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:22.262028 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:22.263622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:22.263622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:22.263622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:22.263622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:22.263622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:22.263622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:22.263622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:22.263622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:22.263622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:22.263622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:22.263622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 15:00:22.263622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:00:22.263622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 80/168] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:00:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:00:22.263622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a40e87e-5581-482d-9535-806383ef0b9d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:22.297077 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89937971-b2ea-4803-82cc-50508e7672bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:22.298817 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89937971-b2ea-4803-82cc-50508e7672bb None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:22.299178 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89937971-b2ea-4803-82cc-50508e7672bb None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:22.299463 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89937971-b2ea-4803-82cc-50508e7672bb None admin] PATH_INFO: `/v3/users/9c29995c0b564cd7879c58c75dc099cb` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:22.302172 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89937971-b2ea-4803-82cc-50508e7672bb None admin] RBAC: Authorizing `identity:get_user(user_id=9c29995c0b564cd7879c58c75dc099cb)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:22.305791 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89937971-b2ea-4803-82cc-50508e7672bb None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:22.310195 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 89/169] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:00:22 2022] GET /identity/v3/users/9c29995c0b564cd7879c58c75dc099cb => generated 247 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:22.311016 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89937971-b2ea-4803-82cc-50508e7672bb None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:22.425315 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a40e87e-5581-482d-9535-806383ef0b9d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:22.446109 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89937971-b2ea-4803-82cc-50508e7672bb None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:22.466090 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268a5926-4615-43b9-b7c3-a0953ea8ca6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:22.466755 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a5926-4615-43b9-b7c3-a0953ea8ca6e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:22.467064 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a5926-4615-43b9-b7c3-a0953ea8ca6e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:22.467346 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a5926-4615-43b9-b7c3-a0953ea8ca6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:22.511536 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:22.523926 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f392ed-83b5-4892-a84f-69101edc89a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:22.524670 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f392ed-83b5-4892-a84f-69101edc89a3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:22.524961 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f392ed-83b5-4892-a84f-69101edc89a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:22.525236 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f392ed-83b5-4892-a84f-69101edc89a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:22.525993 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f392ed-83b5-4892-a84f-69101edc89a3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:22.526298 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f392ed-83b5-4892-a84f-69101edc89a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:22.527200 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:22.527709 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f392ed-83b5-4892-a84f-69101edc89a3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:22.545026 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 90/170] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 15:00:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:22.545674 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f392ed-83b5-4892-a84f-69101edc89a3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:22.564459 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268a5926-4615-43b9-b7c3-a0953ea8ca6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:22.565959 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268a5926-4615-43b9-b7c3-a0953ea8ca6e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:22.668572 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 81/171] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:00:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 15:00:22.669501 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268a5926-4615-43b9-b7c3-a0953ea8ca6e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:22.761269 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:22.795643 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f392ed-83b5-4892-a84f-69101edc89a3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:22.822183 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268a5926-4615-43b9-b7c3-a0953ea8ca6e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:22.880926 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a012729-9bb3-4b53-a8a8-434925a9a0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:22.881608 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a012729-9bb3-4b53-a8a8-434925a9a0f0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:22.881895 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a012729-9bb3-4b53-a8a8-434925a9a0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:22.882171 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a012729-9bb3-4b53-a8a8-434925a9a0f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:22.884330 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b3a17d-3bb0-4b30-812e-7cb9c623758f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:22.885221 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b3a17d-3bb0-4b30-812e-7cb9c623758f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:22.885502 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b3a17d-3bb0-4b30-812e-7cb9c623758f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:22.885778 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b3a17d-3bb0-4b30-812e-7cb9c623758f None admin] PATH_INFO: `/v3/users/aac9cdff3dc345eaa75a0bec09d65b4c` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:22.888846 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b3a17d-3bb0-4b30-812e-7cb9c623758f None admin] RBAC: Authorizing `identity:get_user(user_id=aac9cdff3dc345eaa75a0bec09d65b4c)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:22.890339 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b3a17d-3bb0-4b30-812e-7cb9c623758f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:22.892184 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 82/172] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:00:22 2022] GET /identity/v3/users/aac9cdff3dc345eaa75a0bec09d65b4c => generated 275 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:22.893070 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b3a17d-3bb0-4b30-812e-7cb9c623758f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:22.998272 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:23.010328 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a012729-9bb3-4b53-a8a8-434925a9a0f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:23.011793 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a012729-9bb3-4b53-a8a8-434925a9a0f0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:23.085024 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b3a17d-3bb0-4b30-812e-7cb9c623758f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:23.145554 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 91/173] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:00:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 600 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 15:00:23.146208 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a012729-9bb3-4b53-a8a8-434925a9a0f0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:23.186279 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7bb2f2-4de0-4b9a-9702-0ac0366f10f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:23.187188 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7bb2f2-4de0-4b9a-9702-0ac0366f10f6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:23.187470 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7bb2f2-4de0-4b9a-9702-0ac0366f10f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:23.187793 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7bb2f2-4de0-4b9a-9702-0ac0366f10f6 None admin] PATH_INFO: `/v3/projects/00203c7cc02341ee8ee7a591095f0c7d` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:23.189030 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7bb2f2-4de0-4b9a-9702-0ac0366f10f6 None admin] RBAC: Authorizing `identity:get_project(project_id=00203c7cc02341ee8ee7a591095f0c7d)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:23.190492 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7bb2f2-4de0-4b9a-9702-0ac0366f10f6 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:23.196486 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 83/174] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:00:22 2022] GET /identity/v3/projects/00203c7cc02341ee8ee7a591095f0c7d => generated 298 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:23.197060 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7bb2f2-4de0-4b9a-9702-0ac0366f10f6 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:23.204436 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:23.244907 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:23.252252 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a012729-9bb3-4b53-a8a8-434925a9a0f0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:23.286506 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9de7207-5bcd-4180-874f-6dcaddc856cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:23.287212 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de7207-5bcd-4180-874f-6dcaddc856cc None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:23.287483 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de7207-5bcd-4180-874f-6dcaddc856cc None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:23.287759 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de7207-5bcd-4180-874f-6dcaddc856cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:23.358599 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9de7207-5bcd-4180-874f-6dcaddc856cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:23.360303 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9de7207-5bcd-4180-874f-6dcaddc856cc None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:23.438919 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7bb2f2-4de0-4b9a-9702-0ac0366f10f6 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:23.465548 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:23.473386 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 92/175] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:00:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 15:00:23.474534 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9de7207-5bcd-4180-874f-6dcaddc856cc None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:23.516543 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57687713-1936-48e1-b8fa-919d86361600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:23.517636 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57687713-1936-48e1-b8fa-919d86361600 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:23.517924 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57687713-1936-48e1-b8fa-919d86361600 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:23.518201 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57687713-1936-48e1-b8fa-919d86361600 None admin] PATH_INFO: `/v3/projects/00203c7cc02341ee8ee7a591095f0c7d` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:23.521084 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57687713-1936-48e1-b8fa-919d86361600 None admin] RBAC: Authorizing `identity:get_project(project_id=00203c7cc02341ee8ee7a591095f0c7d)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:23.522682 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57687713-1936-48e1-b8fa-919d86361600 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:23.524588 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 84/176] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:00:23 2022] GET /identity/v3/projects/00203c7cc02341ee8ee7a591095f0c7d => generated 298 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:23.525633 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57687713-1936-48e1-b8fa-919d86361600 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:23.584266 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9de7207-5bcd-4180-874f-6dcaddc856cc None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:23.619879 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de31a9cc-b7c9-46c6-8ba8-acb8c39e8657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:23.620808 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31a9cc-b7c9-46c6-8ba8-acb8c39e8657 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:23.621238 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31a9cc-b7c9-46c6-8ba8-acb8c39e8657 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:23.621666 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de31a9cc-b7c9-46c6-8ba8-acb8c39e8657 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:23.622649 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de31a9cc-b7c9-46c6-8ba8-acb8c39e8657 None admin] RBAC: Adding query filter params (role.id=ad9498a5e2064a468acc151fb2bce49e, scope.project.id=00203c7cc02341ee8ee7a591095f0c7d, user.id=aac9cdff3dc345eaa75a0bec09d65b4c) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:23.623203 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de31a9cc-b7c9-46c6-8ba8-acb8c39e8657 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:23.624813 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de31a9cc-b7c9-46c6-8ba8-acb8c39e8657 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:23.631877 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 93/177] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:00:23 2022] GET /identity/v3/role_assignments?user.id=aac9cdff3dc345eaa75a0bec09d65b4c&scope.project.id=00203c7cc02341ee8ee7a591095f0c7d&role.id=ad9498a5e2064a468acc151fb2bce49e => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:23.632910 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de31a9cc-b7c9-46c6-8ba8-acb8c39e8657 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:23.655203 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:23.713963 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:23.717078 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57687713-1936-48e1-b8fa-919d86361600 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:23.759297 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de31a9cc-b7c9-46c6-8ba8-acb8c39e8657 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:23.799668 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bc685c-db7f-4072-a35d-c4f868fd6d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:23.800566 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc685c-db7f-4072-a35d-c4f868fd6d32 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:23.800851 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc685c-db7f-4072-a35d-c4f868fd6d32 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:23.801128 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc685c-db7f-4072-a35d-c4f868fd6d32 None admin] PATH_INFO: `/v3/projects/a8102aa3c6634ff6b42c5df957f24182` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:23.801399 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bda7342-c2d4-451d-b830-3b3eefe578fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:23.802412 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bc685c-db7f-4072-a35d-c4f868fd6d32 None admin] RBAC: Authorizing `identity:get_project(project_id=a8102aa3c6634ff6b42c5df957f24182)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:23.802723 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda7342-c2d4-451d-b830-3b3eefe578fa None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:23.803188 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda7342-c2d4-451d-b830-3b3eefe578fa None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:23.803608 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bda7342-c2d4-451d-b830-3b3eefe578fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:23.803945 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bc685c-db7f-4072-a35d-c4f868fd6d32 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:23.804773 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bda7342-c2d4-451d-b830-3b3eefe578fa None admin] RBAC: Adding query filter params (role.id=759d845eb6284e42931b4265af448ebc, scope.project.id=00203c7cc02341ee8ee7a591095f0c7d, user.id=9c29995c0b564cd7879c58c75dc099cb) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:23.805231 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bda7342-c2d4-451d-b830-3b3eefe578fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:23.806863 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bda7342-c2d4-451d-b830-3b3eefe578fa None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:23.808365 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 85/178] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:00:23 2022] GET /identity/v3/projects/a8102aa3c6634ff6b42c5df957f24182 => generated 312 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:23.815260 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 94/179] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:00:23 2022] GET /identity/v3/role_assignments?user.id=9c29995c0b564cd7879c58c75dc099cb&scope.project.id=00203c7cc02341ee8ee7a591095f0c7d&role.id=759d845eb6284e42931b4265af448ebc => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:23.816654 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bc685c-db7f-4072-a35d-c4f868fd6d32 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:23.986192 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bc685c-db7f-4072-a35d-c4f868fd6d32 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:24.030621 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fed778d-ad42-42fc-baef-452f2a179473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:24.031320 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed778d-ad42-42fc-baef-452f2a179473 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:24.031606 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed778d-ad42-42fc-baef-452f2a179473 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:24.031882 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed778d-ad42-42fc-baef-452f2a179473 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:24.033082 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fed778d-ad42-42fc-baef-452f2a179473 None admin] RBAC: Adding query filter params (role.id=8442c42e847a4b38b76f3a7fbe9ada85, scope.project.id=a8102aa3c6634ff6b42c5df957f24182, user.id=aac9cdff3dc345eaa75a0bec09d65b4c) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:24.033442 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fed778d-ad42-42fc-baef-452f2a179473 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:24.039134 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fed778d-ad42-42fc-baef-452f2a179473 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:24.046004 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 86/180] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:00:23 2022] GET /identity/v3/role_assignments?user.id=aac9cdff3dc345eaa75a0bec09d65b4c&scope.project.id=a8102aa3c6634ff6b42c5df957f24182&role.id=8442c42e847a4b38b76f3a7fbe9ada85 => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:00:24.631881 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5bfe6-3dad-4fb1-bce2-250074649095 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:24.632583 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5bfe6-3dad-4fb1-bce2-250074649095 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:24.633077 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5bfe6-3dad-4fb1-bce2-250074649095 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:24.635079 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 95/181] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:24.640730 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dd6f44-ca1a-4bfd-9a26-18012b770569 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:24.641259 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dd6f44-ca1a-4bfd-9a26-18012b770569 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:24.641798 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dd6f44-ca1a-4bfd-9a26-18012b770569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:24.647319 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:24.662013 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-98dd6f44-ca1a-4bfd-9a26-18012b770569 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:24.663249 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-98dd6f44-ca1a-4bfd-9a26-18012b770569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:24.749171 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 87/182] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 15:00:24.776508 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5bfe6-3dad-4fb1-bce2-250074649095 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:24.898097 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5bfe6-3dad-4fb1-bce2-250074649095 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:24.921804 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406698f-9d13-4f3e-bdf8-cba43736c72b None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:24.922110 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406698f-9d13-4f3e-bdf8-cba43736c72b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:24.922371 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3406698f-9d13-4f3e-bdf8-cba43736c72b None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:24.924500 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 88/183] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:24.925250 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:24.936441 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472279a-b784-4961-8a64-4bce3e0ae0a3 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:24.937157 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472279a-b784-4961-8a64-4bce3e0ae0a3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:24.937749 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3472279a-b784-4961-8a64-4bce3e0ae0a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:24.940040 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0937f1a-66e0-4dc0-9736-8e29c60abd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:24.940848 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0937f1a-66e0-4dc0-9736-8e29c60abd5c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:24.941124 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0937f1a-66e0-4dc0-9736-8e29c60abd5c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:24.941401 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0937f1a-66e0-4dc0-9736-8e29c60abd5c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:24.954493 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0937f1a-66e0-4dc0-9736-8e29c60abd5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:24.958407 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3472279a-b784-4961-8a64-4bce3e0ae0a3 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:24.959853 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3472279a-b784-4961-8a64-4bce3e0ae0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:24.962938 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-e0937f1a-66e0-4dc0-9736-8e29c60abd5c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:00:24.962938 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:24.962938 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:24.962938 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:24.962938 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:24.962938 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:24.962938 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:24.962938 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:24.962938 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:24.962938 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:24.962938 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:24.962938 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:24.962938 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:24.962938 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:24.962938 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:24.962938 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:24.963823 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:24.963823 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:24.963823 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:24.963823 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:24.963823 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:24.963823 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:24.963823 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:24.963823 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:24.963823 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:24.963823 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:24.963823 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:24.963823 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:00:24.963823 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:24.963823 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:24.963823 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:24.963823 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:24.964623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:24.964623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:24.964623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:24.964623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:24.964623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:24.964623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:24.964623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:24.964623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:24.964623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:24.964623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:24.964623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:00:24.964623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:00:24.964623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 96/184] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 15:00:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:00:24.970304 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0937f1a-66e0-4dc0-9736-8e29c60abd5c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:25.041202 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 89/185] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 15:00:25.045658 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c08be1-7e7a-42ee-bc57-22c72dc344c9 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:25.045658 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c08be1-7e7a-42ee-bc57-22c72dc344c9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:25.045658 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c08be1-7e7a-42ee-bc57-22c72dc344c9 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:25.045658 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 90/186] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:25.056384 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb06676-aa60-4563-bb63-175c827cdb73 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:25.059071 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb06676-aa60-4563-bb63-175c827cdb73 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:25.059620 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb06676-aa60-4563-bb63-175c827cdb73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:25.078024 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-fdb06676-aa60-4563-bb63-175c827cdb73 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:25.080010 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fdb06676-aa60-4563-bb63-175c827cdb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:25.089798 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0937f1a-66e0-4dc0-9736-8e29c60abd5c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:25.131550 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1494adf-5c73-4d7e-9595-7d88c479270a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:25.132227 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1494adf-5c73-4d7e-9595-7d88c479270a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:25.132512 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1494adf-5c73-4d7e-9595-7d88c479270a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:25.132790 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1494adf-5c73-4d7e-9595-7d88c479270a None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:25.133540 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1494adf-5c73-4d7e-9595-7d88c479270a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:25.133892 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1494adf-5c73-4d7e-9595-7d88c479270a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:25.135278 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1494adf-5c73-4d7e-9595-7d88c479270a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:25.140898 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:25.145350 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 97/187] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:00:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:00:25.149938 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1494adf-5c73-4d7e-9595-7d88c479270a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:25.167044 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 91/188] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:00:25.167044 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb06676-aa60-4563-bb63-175c827cdb73 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:25.197584 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:25.256193 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1494adf-5c73-4d7e-9595-7d88c479270a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:25.275545 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb06676-aa60-4563-bb63-175c827cdb73 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:25.292486 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28726a3-4a25-475d-996c-1b4d7a0cd673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:25.293298 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28726a3-4a25-475d-996c-1b4d7a0cd673 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:25.293585 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28726a3-4a25-475d-996c-1b4d7a0cd673 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:25.293860 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28726a3-4a25-475d-996c-1b4d7a0cd673 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:25.304472 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28726a3-4a25-475d-996c-1b4d7a0cd673 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:25.311654 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4578faf5-d367-4b92-8f7a-3bdfc17316b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:25.312531 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-a28726a3-4a25-475d-996c-1b4d7a0cd673 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 15:00:25.312531 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:25.312531 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:25.312531 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:25.312531 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:25.312531 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:25.312531 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:25.312531 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:25.312531 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:25.312531 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:25.312531 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:25.312531 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:25.312531 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:25.312531 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:25.312531 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:25.312531 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:25.314050 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:25.314050 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:25.314050 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:25.314050 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:25.314050 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:25.314050 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:25.314050 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:25.314050 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:25.314050 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:25.314050 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:25.314050 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:25.314050 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:00:25.314050 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:25.314050 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:25.314050 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:25.314050 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:25.314886 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:25.314886 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:25.314886 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:25.314886 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:25.314886 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:25.314886 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:25.314886 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:25.314886 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:25.314886 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:25.314886 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:25.314886 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 15:00:25.314886 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:00:25.315503 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578faf5-d367-4b92-8f7a-3bdfc17316b4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:25.315503 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578faf5-d367-4b92-8f7a-3bdfc17316b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:25.315503 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578faf5-d367-4b92-8f7a-3bdfc17316b4 None admin] PATH_INFO: `/v3/users/e22d1b017e4744cda46d80abfdbb947c` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:25.315693 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 98/189] 10.222.0.50 () {56 vars in 1114 bytes} [Fri May 27 15:00:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:00:25.315693 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28726a3-4a25-475d-996c-1b4d7a0cd673 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:25.330193 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4578faf5-d367-4b92-8f7a-3bdfc17316b4 None admin] RBAC: Authorizing `identity:get_user(user_id=e22d1b017e4744cda46d80abfdbb947c)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:25.331727 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4578faf5-d367-4b92-8f7a-3bdfc17316b4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:25.337956 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 92/190] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:00:25 2022] GET /identity/v3/users/e22d1b017e4744cda46d80abfdbb947c => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:25.337956 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4578faf5-d367-4b92-8f7a-3bdfc17316b4 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:25.390297 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:25.444129 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28726a3-4a25-475d-996c-1b4d7a0cd673 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:25.447456 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:25.448383 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4578faf5-d367-4b92-8f7a-3bdfc17316b4 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:25.481512 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4273a246-0521-4c19-a2d2-dc3c7d659c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:25.482360 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4273a246-0521-4c19-a2d2-dc3c7d659c1d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:25.482594 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4273a246-0521-4c19-a2d2-dc3c7d659c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:25.482881 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4273a246-0521-4c19-a2d2-dc3c7d659c1d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:25.484696 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d6862-809e-454f-b1d4-47d076952193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:25.486132 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d6862-809e-454f-b1d4-47d076952193 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:25.486546 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d6862-809e-454f-b1d4-47d076952193 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:25.486828 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d6862-809e-454f-b1d4-47d076952193 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:25.487601 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d6862-809e-454f-b1d4-47d076952193 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:25.487913 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d6862-809e-454f-b1d4-47d076952193 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:25.489287 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d6862-809e-454f-b1d4-47d076952193 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:25.494423 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4273a246-0521-4c19-a2d2-dc3c7d659c1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:25.497954 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 93/191] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:00:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:25.498830 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d6862-809e-454f-b1d4-47d076952193 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:25.507217 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-4273a246-0521-4c19-a2d2-dc3c7d659c1d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 15:00:25.507217 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:25.507217 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:25.507217 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:25.507217 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:25.507217 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:25.507217 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:25.507217 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:25.507217 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:25.507217 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:25.507217 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:25.507217 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:25.507217 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:25.507217 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:25.507217 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:25.507217 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:25.508152 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:25.508152 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:25.508152 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:25.508152 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:25.508152 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:25.508152 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:25.508152 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:25.508152 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:25.508152 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:25.508152 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:25.508152 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:25.508152 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:00:25.508152 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:25.508152 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:25.508152 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:25.508152 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:25.508962 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:25.508962 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:25.508962 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:25.508962 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:25.508962 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:25.508962 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:25.508962 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:25.508962 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:25.508962 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:25.508962 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:25.508962 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 15:00:25.508962 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:00:25.508962 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 99/192] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:00:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:00:25.509615 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4273a246-0521-4c19-a2d2-dc3c7d659c1d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:25.606672 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d6862-809e-454f-b1d4-47d076952193 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:25.619710 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:25.621821 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4273a246-0521-4c19-a2d2-dc3c7d659c1d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:25.640171 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c550a6b3-27dc-4698-896b-d81c8ed32160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:25.640819 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550a6b3-27dc-4698-896b-d81c8ed32160 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:25.641112 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550a6b3-27dc-4698-896b-d81c8ed32160 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:25.641429 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c550a6b3-27dc-4698-896b-d81c8ed32160 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:25.655032 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f751fb57-807e-4e87-ba69-c6eada164daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:25.655912 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751fb57-807e-4e87-ba69-c6eada164daa None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:25.656195 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751fb57-807e-4e87-ba69-c6eada164daa None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:25.656471 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751fb57-807e-4e87-ba69-c6eada164daa None admin] PATH_INFO: `/v3/users/e22d1b017e4744cda46d80abfdbb947c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:25.658184 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f751fb57-807e-4e87-ba69-c6eada164daa None admin] RBAC: Authorizing `identity:get_user(user_id=e22d1b017e4744cda46d80abfdbb947c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:25.659683 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f751fb57-807e-4e87-ba69-c6eada164daa None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:25.661315 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 100/193] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:00:25 2022] GET /identity/v3/users/e22d1b017e4744cda46d80abfdbb947c => generated 283 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:25.662080 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f751fb57-807e-4e87-ba69-c6eada164daa None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:25.693101 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:25.710194 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c550a6b3-27dc-4698-896b-d81c8ed32160 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:25.711620 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c550a6b3-27dc-4698-896b-d81c8ed32160 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:25.761094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f751fb57-807e-4e87-ba69-c6eada164daa None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:25.793319 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af2e71d-7262-4dd9-8f03-9d1a9c1ca60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:25.793978 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af2e71d-7262-4dd9-8f03-9d1a9c1ca60a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:25.794261 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af2e71d-7262-4dd9-8f03-9d1a9c1ca60a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:25.794535 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af2e71d-7262-4dd9-8f03-9d1a9c1ca60a None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:25.795334 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af2e71d-7262-4dd9-8f03-9d1a9c1ca60a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:25.795637 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af2e71d-7262-4dd9-8f03-9d1a9c1ca60a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:25.796999 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af2e71d-7262-4dd9-8f03-9d1a9c1ca60a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:25.805444 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 101/194] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 15:00:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:25.806648 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8034860-5808-42af-84e8-249801229103 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:25.806963 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8034860-5808-42af-84e8-249801229103 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:25.807241 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8034860-5808-42af-84e8-249801229103 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:25.812700 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 94/195] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:00:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 15:00:25.812829 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 102/196] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:25.812903 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c550a6b3-27dc-4698-896b-d81c8ed32160 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:25.812993 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8034860-5808-42af-84e8-249801229103 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:25.825691 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:25.904251 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:25.910034 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c550a6b3-27dc-4698-896b-d81c8ed32160 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:25.912289 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8034860-5808-42af-84e8-249801229103 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:25.943997 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a6c265-32c8-4b5d-bae7-38070b4dd8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:25.944640 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6c265-32c8-4b5d-bae7-38070b4dd8fd None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:25.944922 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6c265-32c8-4b5d-bae7-38070b4dd8fd None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:25.945194 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6c265-32c8-4b5d-bae7-38070b4dd8fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:25.945747 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdff9004-b383-4f4a-bc95-96fb9f365879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:25.946612 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff9004-b383-4f4a-bc95-96fb9f365879 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:25.946918 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff9004-b383-4f4a-bc95-96fb9f365879 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:25.947195 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdff9004-b383-4f4a-bc95-96fb9f365879 None admin] PATH_INFO: `/v3/users/6f93dac3329a47ca90ee32eed0f164e7` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:25.948381 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdff9004-b383-4f4a-bc95-96fb9f365879 None admin] RBAC: Authorizing `identity:get_user(user_id=6f93dac3329a47ca90ee32eed0f164e7)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:25.949817 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdff9004-b383-4f4a-bc95-96fb9f365879 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:25.951462 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 103/197] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:00:25 2022] GET /identity/v3/users/6f93dac3329a47ca90ee32eed0f164e7 => generated 289 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:25.956369 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdff9004-b383-4f4a-bc95-96fb9f365879 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:26.011999 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a6c265-32c8-4b5d-bae7-38070b4dd8fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:26.013474 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a6c265-32c8-4b5d-bae7-38070b4dd8fd None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:26.026977 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:26.049663 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdff9004-b383-4f4a-bc95-96fb9f365879 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:26.081904 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d65e95-3068-43be-819c-0110b26b96e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:26.082753 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d65e95-3068-43be-819c-0110b26b96e4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:26.083060 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d65e95-3068-43be-819c-0110b26b96e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:26.083333 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d65e95-3068-43be-819c-0110b26b96e4 None admin] PATH_INFO: `/v3/projects/65801518f2424895ab7af78ae7acb341` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:26.084497 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d65e95-3068-43be-819c-0110b26b96e4 None admin] RBAC: Authorizing `identity:get_project(project_id=65801518f2424895ab7af78ae7acb341)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:26.085914 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d65e95-3068-43be-819c-0110b26b96e4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:26.087891 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 104/198] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:00:25 2022] GET /identity/v3/projects/65801518f2424895ab7af78ae7acb341 => generated 302 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:26.089153 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1126e-7f2f-406f-b778-aefd867848ad None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:26.089436 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1126e-7f2f-406f-b778-aefd867848ad None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:26.089852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1126e-7f2f-406f-b778-aefd867848ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:26.105823 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6ca1126e-7f2f-406f-b778-aefd867848ad None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:26.107025 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6ca1126e-7f2f-406f-b778-aefd867848ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:26.112325 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 95/199] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:00:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 15:00:26.112325 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a6c265-32c8-4b5d-bae7-38070b4dd8fd None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:26.134266 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:26.181902 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 105/200] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:00:26.182750 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca1126e-7f2f-406f-b778-aefd867848ad None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:26.215002 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a6c265-32c8-4b5d-bae7-38070b4dd8fd None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:26.228189 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:26.249133 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a197fbe3-ff85-4f17-90ed-ecb8a8f98640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:26.249866 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197fbe3-ff85-4f17-90ed-ecb8a8f98640 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:26.250077 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197fbe3-ff85-4f17-90ed-ecb8a8f98640 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:26.250334 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a197fbe3-ff85-4f17-90ed-ecb8a8f98640 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:26.282148 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca1126e-7f2f-406f-b778-aefd867848ad None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:26.315569 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbee5e0d-e630-4ee9-89b4-6a00e245d239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:26.316229 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee5e0d-e630-4ee9-89b4-6a00e245d239 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:26.316513 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee5e0d-e630-4ee9-89b4-6a00e245d239 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:26.316790 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee5e0d-e630-4ee9-89b4-6a00e245d239 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:26.317615 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbee5e0d-e630-4ee9-89b4-6a00e245d239 None admin] RBAC: Adding query filter params (role.id=759d845eb6284e42931b4265af448ebc, scope.project.id=65801518f2424895ab7af78ae7acb341, user.id=e22d1b017e4744cda46d80abfdbb947c) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:26.317933 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbee5e0d-e630-4ee9-89b4-6a00e245d239 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:26.318100 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a197fbe3-ff85-4f17-90ed-ecb8a8f98640 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:26.319416 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbee5e0d-e630-4ee9-89b4-6a00e245d239 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:26.319604 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a197fbe3-ff85-4f17-90ed-ecb8a8f98640 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:26.325217 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 106/201] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:00:26 2022] GET /identity/v3/role_assignments?user.id=e22d1b017e4744cda46d80abfdbb947c&scope.project.id=65801518f2424895ab7af78ae7acb341&role.id=759d845eb6284e42931b4265af448ebc => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:26.326000 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbee5e0d-e630-4ee9-89b4-6a00e245d239 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:26.354038 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:26.437967 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbee5e0d-e630-4ee9-89b4-6a00e245d239 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:26.442468 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 96/202] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:00:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 15:00:26.443627 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a197fbe3-ff85-4f17-90ed-ecb8a8f98640 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:26.459587 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:26.479268 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f870691-0cf4-47f6-9017-ef58f3f175f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:26.480151 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f870691-0cf4-47f6-9017-ef58f3f175f4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:26.480434 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f870691-0cf4-47f6-9017-ef58f3f175f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:26.480724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f870691-0cf4-47f6-9017-ef58f3f175f4 None admin] PATH_INFO: `/v3/projects/65801518f2424895ab7af78ae7acb341` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:26.481986 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f870691-0cf4-47f6-9017-ef58f3f175f4 None admin] RBAC: Authorizing `identity:get_project(project_id=65801518f2424895ab7af78ae7acb341)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:26.483490 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f870691-0cf4-47f6-9017-ef58f3f175f4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:26.485974 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 107/203] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:00:26 2022] GET /identity/v3/projects/65801518f2424895ab7af78ae7acb341 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:26.491881 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f870691-0cf4-47f6-9017-ef58f3f175f4 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:26.555997 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a197fbe3-ff85-4f17-90ed-ecb8a8f98640 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:26.591598 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf8d14b-fb3b-4847-b1ec-84762cf7eeb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:26.592394 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8d14b-fb3b-4847-b1ec-84762cf7eeb0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:26.592680 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8d14b-fb3b-4847-b1ec-84762cf7eeb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:26.592957 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8d14b-fb3b-4847-b1ec-84762cf7eeb0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:26.594126 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf8d14b-fb3b-4847-b1ec-84762cf7eeb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:26.596158 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf8d14b-fb3b-4847-b1ec-84762cf7eeb0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:26.597801 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 97/204] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:00:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:26.603182 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf8d14b-fb3b-4847-b1ec-84762cf7eeb0 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:26.603717 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:26.614974 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f870691-0cf4-47f6-9017-ef58f3f175f4 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:26.654768 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b4ae19-fc61-4c9c-9fd1-fa864106259e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:26.655668 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4ae19-fc61-4c9c-9fd1-fa864106259e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:26.655951 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4ae19-fc61-4c9c-9fd1-fa864106259e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:26.656228 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4ae19-fc61-4c9c-9fd1-fa864106259e None admin] PATH_INFO: `/v3/projects/00203c7cc02341ee8ee7a591095f0c7d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:26.658977 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b4ae19-fc61-4c9c-9fd1-fa864106259e None admin] RBAC: Authorizing `identity:get_project(project_id=00203c7cc02341ee8ee7a591095f0c7d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:26.660919 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b4ae19-fc61-4c9c-9fd1-fa864106259e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:26.663524 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 108/205] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:00:26 2022] GET /identity/v3/projects/00203c7cc02341ee8ee7a591095f0c7d => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:26.664319 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b4ae19-fc61-4c9c-9fd1-fa864106259e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:26.690432 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:26.715391 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf8d14b-fb3b-4847-b1ec-84762cf7eeb0 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:26.754441 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06d18d8-31e8-42af-876f-67e78789e056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:26.755126 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d18d8-31e8-42af-876f-67e78789e056 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:26.755410 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d18d8-31e8-42af-876f-67e78789e056 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:26.755699 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d18d8-31e8-42af-876f-67e78789e056 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:26.756518 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06d18d8-31e8-42af-876f-67e78789e056 None admin] RBAC: Adding query filter params (role.id=ad9498a5e2064a468acc151fb2bce49e, scope.project.id=65801518f2424895ab7af78ae7acb341, user.id=e22d1b017e4744cda46d80abfdbb947c) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:26.756830 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06d18d8-31e8-42af-876f-67e78789e056 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:26.758349 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06d18d8-31e8-42af-876f-67e78789e056 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:26.764206 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 98/206] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:00:26 2022] GET /identity/v3/role_assignments?user.id=e22d1b017e4744cda46d80abfdbb947c&scope.project.id=65801518f2424895ab7af78ae7acb341&role.id=ad9498a5e2064a468acc151fb2bce49e => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:26.768766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b4ae19-fc61-4c9c-9fd1-fa864106259e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:26.773007 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06d18d8-31e8-42af-876f-67e78789e056 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:26.809220 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28c4000-c62f-4700-93ed-874110381c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:26.809795 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c4000-c62f-4700-93ed-874110381c57 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:26.810147 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c4000-c62f-4700-93ed-874110381c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:26.810426 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28c4000-c62f-4700-93ed-874110381c57 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:26.811248 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28c4000-c62f-4700-93ed-874110381c57 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:26.812897 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28c4000-c62f-4700-93ed-874110381c57 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:26.844210 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 109/207] 10.222.0.50 () {60 vars in 1132 bytes} [Fri May 27 15:00:26 2022] POST /identity/v3/users => generated 297 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:00:26.927315 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06d18d8-31e8-42af-876f-67e78789e056 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:26.969518 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c45a01-178a-4fbb-aa45-a7aaf7d3b5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:26.970747 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c45a01-178a-4fbb-aa45-a7aaf7d3b5a6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:26.971269 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c45a01-178a-4fbb-aa45-a7aaf7d3b5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:26.971742 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c45a01-178a-4fbb-aa45-a7aaf7d3b5a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:26.972861 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c45a01-178a-4fbb-aa45-a7aaf7d3b5a6 None admin] RBAC: Adding query filter params (role.id=1b313eb2327b44b7a11d92914db65bb9, scope.project.id=00203c7cc02341ee8ee7a591095f0c7d, user.id=6f93dac3329a47ca90ee32eed0f164e7) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:26.973376 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c45a01-178a-4fbb-aa45-a7aaf7d3b5a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:26.975088 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c45a01-178a-4fbb-aa45-a7aaf7d3b5a6 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:26.983143 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 99/208] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:00:26 2022] GET /identity/v3/role_assignments?user.id=6f93dac3329a47ca90ee32eed0f164e7&scope.project.id=00203c7cc02341ee8ee7a591095f0c7d&role.id=1b313eb2327b44b7a11d92914db65bb9 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:28.574448 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6988e-1aaa-4fdc-aee6-42c8edb807ef None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:28.574990 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6988e-1aaa-4fdc-aee6-42c8edb807ef None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:28.575068 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c6988e-1aaa-4fdc-aee6-42c8edb807ef None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:28.576412 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 110/209] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:28.582348 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d334187-cdbd-4604-9d22-865669eee810 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:28.582828 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d334187-cdbd-4604-9d22-865669eee810 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:28.582941 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d334187-cdbd-4604-9d22-865669eee810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:28.600854 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7d334187-cdbd-4604-9d22-865669eee810 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:28.602031 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7d334187-cdbd-4604-9d22-865669eee810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:28.604025 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:28.672636 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 100/210] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:00:28.697006 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6988e-1aaa-4fdc-aee6-42c8edb807ef None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:28.720934 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:28.811858 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c6988e-1aaa-4fdc-aee6-42c8edb807ef None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:28.850666 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66d5c2-deb0-4fd9-ac82-ac9d0aee918e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:28.851435 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66d5c2-deb0-4fd9-ac82-ac9d0aee918e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:28.851713 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66d5c2-deb0-4fd9-ac82-ac9d0aee918e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:28.852037 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c66d5c2-deb0-4fd9-ac82-ac9d0aee918e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:28.865346 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c66d5c2-deb0-4fd9-ac82-ac9d0aee918e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:28.873276 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-7c66d5c2-deb0-4fd9-ac82-ac9d0aee918e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:00:28.873276 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:28.873276 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:28.873276 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:28.873276 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:28.873276 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:28.873276 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:28.873276 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:28.873276 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:28.873276 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:28.873276 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:28.873276 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:28.873276 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:28.873276 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:28.873276 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:28.873276 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:28.874226 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:28.874226 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:28.874226 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:28.874226 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:28.874226 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:28.874226 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:28.874226 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:28.874226 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:28.874226 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:28.874226 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:28.874226 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:28.874226 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:00:28.874226 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:28.874226 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:28.874226 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:28.874226 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:28.875050 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:28.875050 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:28.875050 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:28.875050 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:28.875050 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:28.875050 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:28.875050 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:28.875050 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:28.875050 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:28.875050 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:28.875050 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:00:28.875050 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:00:28.875050 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 111/211] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 15:00:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:00:28.880976 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d334187-cdbd-4604-9d22-865669eee810 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:28.981355 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d334187-cdbd-4604-9d22-865669eee810 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:29.017647 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5076a105-d3f5-4d44-8390-3cadc73d637e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:29.018326 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076a105-d3f5-4d44-8390-3cadc73d637e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:29.018609 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076a105-d3f5-4d44-8390-3cadc73d637e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:29.018910 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076a105-d3f5-4d44-8390-3cadc73d637e None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:29.019717 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5076a105-d3f5-4d44-8390-3cadc73d637e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:29.020030 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5076a105-d3f5-4d44-8390-3cadc73d637e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:29.021426 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5076a105-d3f5-4d44-8390-3cadc73d637e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:29.026873 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:29.030468 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 101/212] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:00:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:29.037096 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66d5c2-deb0-4fd9-ac82-ac9d0aee918e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:29.038326 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e8003-9763-4d6a-9a74-09e0652156ab None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:29.038622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e8003-9763-4d6a-9a74-09e0652156ab None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:29.038945 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805e8003-9763-4d6a-9a74-09e0652156ab None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:29.040313 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 102/213] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 15:00:29.045478 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e40ee-bfed-44d9-a1d3-fbd8e0708553 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:29.045767 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e40ee-bfed-44d9-a1d3-fbd8e0708553 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:29.046046 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858e40ee-bfed-44d9-a1d3-fbd8e0708553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:29.064710 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-858e40ee-bfed-44d9-a1d3-fbd8e0708553 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:29.066531 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-858e40ee-bfed-44d9-a1d3-fbd8e0708553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:29.120549 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:29.158915 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 103/214] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 15:00:29.161873 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c66d5c2-deb0-4fd9-ac82-ac9d0aee918e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:29.187328 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e40ee-bfed-44d9-a1d3-fbd8e0708553 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:29.210535 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539085a6-6554-4bf6-b4aa-fa534af9042a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:29.211841 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539085a6-6554-4bf6-b4aa-fa534af9042a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:29.212336 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539085a6-6554-4bf6-b4aa-fa534af9042a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:29.213060 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539085a6-6554-4bf6-b4aa-fa534af9042a None admin] PATH_INFO: `/v3/users/e22d1b017e4744cda46d80abfdbb947c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:29.214510 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539085a6-6554-4bf6-b4aa-fa534af9042a None admin] RBAC: Authorizing `identity:get_user(user_id=e22d1b017e4744cda46d80abfdbb947c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:29.216170 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539085a6-6554-4bf6-b4aa-fa534af9042a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:29.218342 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 112/215] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:00:29 2022] GET /identity/v3/users/e22d1b017e4744cda46d80abfdbb947c => generated 283 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:29.224511 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539085a6-6554-4bf6-b4aa-fa534af9042a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:29.293973 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858e40ee-bfed-44d9-a1d3-fbd8e0708553 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:29.324801 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:29.331251 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539085a6-6554-4bf6-b4aa-fa534af9042a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:29.332423 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c2676b-e8a5-4e80-9fc6-9174c4e4f183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:29.333249 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2676b-e8a5-4e80-9fc6-9174c4e4f183 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:29.333648 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2676b-e8a5-4e80-9fc6-9174c4e4f183 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:29.333789 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c2676b-e8a5-4e80-9fc6-9174c4e4f183 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:29.346218 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c2676b-e8a5-4e80-9fc6-9174c4e4f183 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:29.354471 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-b2c2676b-e8a5-4e80-9fc6-9174c4e4f183 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 15:00:29.354471 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:29.354471 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:29.354471 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:29.354471 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:29.354471 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:29.354471 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:29.354471 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:29.354471 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:29.354471 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:29.354471 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:29.354471 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:29.354471 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:29.354471 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:29.354471 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:29.354471 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:29.355567 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:29.355567 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:29.355567 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:29.355567 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:29.355567 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:29.355567 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:29.355567 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:29.355567 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:29.355567 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:29.355567 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:29.355567 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:29.355567 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:00:29.355567 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:29.355567 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:29.355567 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:29.355567 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:29.356364 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:29.356364 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:29.356364 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:29.356364 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:29.356364 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:29.356364 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:29.356364 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:29.356364 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:29.356364 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:29.356364 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:29.356364 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 15:00:29.356364 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:00:29.356364 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 104/216] 10.222.0.50 () {56 vars in 1114 bytes} [Fri May 27 15:00:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:00:29.357039 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231fade-b202-4088-9d33-94602adf0c49 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:29.357301 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231fade-b202-4088-9d33-94602adf0c49 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:29.357518 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a231fade-b202-4088-9d33-94602adf0c49 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:29.358762 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 105/217] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:29.359874 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f973e496-159b-4cbc-8bf8-4f5dc09e2442 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:29.360748 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f973e496-159b-4cbc-8bf8-4f5dc09e2442 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:29.361013 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f973e496-159b-4cbc-8bf8-4f5dc09e2442 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:29.362433 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 106/218] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:29.363058 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f973e496-159b-4cbc-8bf8-4f5dc09e2442 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:29.373777 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd23c5a8-f64c-4e17-88cd-408246980529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:29.374573 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23c5a8-f64c-4e17-88cd-408246980529 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:29.374892 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23c5a8-f64c-4e17-88cd-408246980529 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:29.375152 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23c5a8-f64c-4e17-88cd-408246980529 None admin] PATH_INFO: `/v3/projects/65801518f2424895ab7af78ae7acb341` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:29.376356 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd23c5a8-f64c-4e17-88cd-408246980529 None admin] RBAC: Authorizing `identity:get_project(project_id=65801518f2424895ab7af78ae7acb341)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:29.377829 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd23c5a8-f64c-4e17-88cd-408246980529 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:29.379562 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 113/219] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:00:29 2022] GET /identity/v3/projects/65801518f2424895ab7af78ae7acb341 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:00:29.384216 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590de116-016f-4bcf-aeac-a114bdbc9235 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:29.384513 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590de116-016f-4bcf-aeac-a114bdbc9235 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:29.384794 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590de116-016f-4bcf-aeac-a114bdbc9235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:29.400749 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-590de116-016f-4bcf-aeac-a114bdbc9235 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:29.401915 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-590de116-016f-4bcf-aeac-a114bdbc9235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:29.409864 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:29.466264 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f973e496-159b-4cbc-8bf8-4f5dc09e2442 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:29.471928 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 114/220] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:00:29.473192 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd58c60-df52-4ae9-91d5-08a4c0c3c031 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:29.473487 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd58c60-df52-4ae9-91d5-08a4c0c3c031 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:29.473820 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd58c60-df52-4ae9-91d5-08a4c0c3c031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:29.492729 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-ecd58c60-df52-4ae9-91d5-08a4c0c3c031 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:29.493894 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ecd58c60-df52-4ae9-91d5-08a4c0c3c031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:29.501195 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff5e23-ea70-40b0-8f42-8fefa9c4d3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:29.501848 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff5e23-ea70-40b0-8f42-8fefa9c4d3e0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:29.502132 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff5e23-ea70-40b0-8f42-8fefa9c4d3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:29.502411 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff5e23-ea70-40b0-8f42-8fefa9c4d3e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:29.503185 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ff5e23-ea70-40b0-8f42-8fefa9c4d3e0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:29.503494 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ff5e23-ea70-40b0-8f42-8fefa9c4d3e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:29.504874 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ff5e23-ea70-40b0-8f42-8fefa9c4d3e0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:29.512747 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 107/221] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:00:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:29.513517 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff5e23-ea70-40b0-8f42-8fefa9c4d3e0 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:29.569457 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:29.576110 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 115/222] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:00:29.577240 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd58c60-df52-4ae9-91d5-08a4c0c3c031 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:29.619297 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff5e23-ea70-40b0-8f42-8fefa9c4d3e0 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:29.647140 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:29.653142 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58ba2d6-c7d7-4ab0-ba9b-57e197fe6dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:29.654025 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ba2d6-c7d7-4ab0-ba9b-57e197fe6dfe None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:29.654305 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ba2d6-c7d7-4ab0-ba9b-57e197fe6dfe None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:29.654582 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58ba2d6-c7d7-4ab0-ba9b-57e197fe6dfe None admin] PATH_INFO: `/v3/projects/65801518f2424895ab7af78ae7acb341/users/e22d1b017e4744cda46d80abfdbb947c/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:29.656560 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58ba2d6-c7d7-4ab0-ba9b-57e197fe6dfe None admin] RBAC: Authorizing `identity:create_grant(project_id=65801518f2424895ab7af78ae7acb341, user_id=e22d1b017e4744cda46d80abfdbb947c, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:29.658616 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58ba2d6-c7d7-4ab0-ba9b-57e197fe6dfe None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:29.670649 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 108/223] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 15:00:29 2022] PUT /identity/v3/projects/65801518f2424895ab7af78ae7acb341/users/e22d1b017e4744cda46d80abfdbb947c/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:00:29.683040 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58ba2d6-c7d7-4ab0-ba9b-57e197fe6dfe None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:29.693235 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd58c60-df52-4ae9-91d5-08a4c0c3c031 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:29.738301 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704389fc-1092-49f9-9eb6-34ee32d68f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:29.739160 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704389fc-1092-49f9-9eb6-34ee32d68f60 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:29.739440 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704389fc-1092-49f9-9eb6-34ee32d68f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:29.739710 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704389fc-1092-49f9-9eb6-34ee32d68f60 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:29.754171 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704389fc-1092-49f9-9eb6-34ee32d68f60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:29.762130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-704389fc-1092-49f9-9eb6-34ee32d68f60 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 15:00:29.762130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:29.762130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:29.762130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:29.762130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:29.762130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:29.762130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:29.762130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:29.762130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:29.762130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:29.762130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:29.762130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:29.762130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:29.762130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:29.762130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:29.762130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:29.763118 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:29.763118 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:29.763118 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:29.763118 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:29.763118 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:29.763118 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:29.763118 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:29.763118 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:29.763118 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:29.763118 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:29.763118 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:29.763118 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:00:29.763118 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:29.763118 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:29.763118 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:29.763118 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:29.763924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:29.763924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:29.763924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:29.763924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:29.763924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:29.763924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:29.763924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:29.763924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:29.763924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:29.763924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:29.763924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 15:00:29.763924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:00:29.763924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 116/224] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:00:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:00:29.764575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704389fc-1092-49f9-9eb6-34ee32d68f60 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:29.794396 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58ba2d6-c7d7-4ab0-ba9b-57e197fe6dfe None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:29.810417 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:29.840022 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf80f33-ea70-448b-8020-4031d8b2150b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:29.840957 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf80f33-ea70-448b-8020-4031d8b2150b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:29.841239 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf80f33-ea70-448b-8020-4031d8b2150b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:29.842033 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf80f33-ea70-448b-8020-4031d8b2150b None admin] PATH_INFO: `/v3/users/e22d1b017e4744cda46d80abfdbb947c` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:29.842840 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf80f33-ea70-448b-8020-4031d8b2150b None admin] RBAC: Authorizing `identity:get_user(user_id=e22d1b017e4744cda46d80abfdbb947c)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:29.844827 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf80f33-ea70-448b-8020-4031d8b2150b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:29.846975 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 109/225] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:00:29 2022] GET /identity/v3/users/e22d1b017e4744cda46d80abfdbb947c => generated 283 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:29.848143 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf80f33-ea70-448b-8020-4031d8b2150b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:29.889568 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704389fc-1092-49f9-9eb6-34ee32d68f60 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:29.926412 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:29.933283 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de920e-4938-4b9a-bb62-98f5159c47d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:29.934121 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de920e-4938-4b9a-bb62-98f5159c47d0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:29.934401 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de920e-4938-4b9a-bb62-98f5159c47d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:29.934672 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0de920e-4938-4b9a-bb62-98f5159c47d0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:29.935921 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0de920e-4938-4b9a-bb62-98f5159c47d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:29.937364 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0de920e-4938-4b9a-bb62-98f5159c47d0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:29.939717 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 117/226] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:00:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:29.940284 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de920e-4938-4b9a-bb62-98f5159c47d0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:29.964416 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf80f33-ea70-448b-8020-4031d8b2150b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:30.001473 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e98c8-3091-4ed4-a33c-856e2a6542fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:30.002762 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e98c8-3091-4ed4-a33c-856e2a6542fc None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:30.003213 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e98c8-3091-4ed4-a33c-856e2a6542fc None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:30.003634 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e98c8-3091-4ed4-a33c-856e2a6542fc None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:30.004545 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071e98c8-3091-4ed4-a33c-856e2a6542fc None admin] RBAC: Adding query filter params (name=member) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:30.005058 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071e98c8-3091-4ed4-a33c-856e2a6542fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:30.006782 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071e98c8-3091-4ed4-a33c-856e2a6542fc None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:30.017432 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 110/227] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 15:00:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:30.018589 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e98c8-3091-4ed4-a33c-856e2a6542fc None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:30.048185 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0de920e-4938-4b9a-bb62-98f5159c47d0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:30.071086 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:30.084432 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026984a7-7c31-4579-96f9-01e78d517e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:30.085297 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026984a7-7c31-4579-96f9-01e78d517e92 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:30.085574 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026984a7-7c31-4579-96f9-01e78d517e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:30.085854 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026984a7-7c31-4579-96f9-01e78d517e92 None admin] PATH_INFO: `/v3/projects/65801518f2424895ab7af78ae7acb341` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:30.087078 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026984a7-7c31-4579-96f9-01e78d517e92 None admin] RBAC: Authorizing `identity:get_project(project_id=65801518f2424895ab7af78ae7acb341)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:30.088785 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026984a7-7c31-4579-96f9-01e78d517e92 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:30.090881 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 118/228] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:00:29 2022] GET /identity/v3/projects/65801518f2424895ab7af78ae7acb341 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:30.091565 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026984a7-7c31-4579-96f9-01e78d517e92 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:30.139677 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e98c8-3091-4ed4-a33c-856e2a6542fc None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:30.160161 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:30.185814 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcabd338-74f6-46ae-952f-9e9116383f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:30.186413 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcabd338-74f6-46ae-952f-9e9116383f33 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:30.186699 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcabd338-74f6-46ae-952f-9e9116383f33 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:30.187008 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcabd338-74f6-46ae-952f-9e9116383f33 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:30.187724 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcabd338-74f6-46ae-952f-9e9116383f33 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:30.189744 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcabd338-74f6-46ae-952f-9e9116383f33 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:30.197766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026984a7-7c31-4579-96f9-01e78d517e92 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:30.210730 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 111/229] 10.222.0.50 () {60 vars in 1132 bytes} [Fri May 27 15:00:30 2022] POST /identity/v3/users => generated 297 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:00:30.211948 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcabd338-74f6-46ae-952f-9e9116383f33 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:30.245423 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c6fb55-b519-4556-98d3-ccdb9bdd153e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:30.246388 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6fb55-b519-4556-98d3-ccdb9bdd153e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:30.246674 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6fb55-b519-4556-98d3-ccdb9bdd153e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:30.246983 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6fb55-b519-4556-98d3-ccdb9bdd153e None admin] PATH_INFO: `/v3/users/dc9d9a4819c74e949d95bae75a1084ed` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:30.263549 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c6fb55-b519-4556-98d3-ccdb9bdd153e None admin] RBAC: Authorizing `identity:get_user(user_id=dc9d9a4819c74e949d95bae75a1084ed)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:30.265346 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c6fb55-b519-4556-98d3-ccdb9bdd153e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:30.267643 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 119/230] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:00:30 2022] GET /identity/v3/users/dc9d9a4819c74e949d95bae75a1084ed => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:30.273124 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c6fb55-b519-4556-98d3-ccdb9bdd153e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:30.360803 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:30.368576 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcabd338-74f6-46ae-952f-9e9116383f33 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:30.382374 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c6fb55-b519-4556-98d3-ccdb9bdd153e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:30.410189 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:30.418938 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07633ad8-a612-44fe-b0be-66976daffb65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:30.419615 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07633ad8-a612-44fe-b0be-66976daffb65 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:30.419809 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c7559c-52ea-4969-9079-8286cb8f22c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:30.419923 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07633ad8-a612-44fe-b0be-66976daffb65 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:30.420162 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07633ad8-a612-44fe-b0be-66976daffb65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:30.421418 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7559c-52ea-4969-9079-8286cb8f22c1 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:30.421907 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7559c-52ea-4969-9079-8286cb8f22c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:30.422399 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c7559c-52ea-4969-9079-8286cb8f22c1 None admin] PATH_INFO: `/v3/projects/65801518f2424895ab7af78ae7acb341/users/e22d1b017e4744cda46d80abfdbb947c/roles/ad9498a5e2064a468acc151fb2bce49e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:30.426875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c7559c-52ea-4969-9079-8286cb8f22c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=65801518f2424895ab7af78ae7acb341, user_id=e22d1b017e4744cda46d80abfdbb947c, role_id=ad9498a5e2064a468acc151fb2bce49e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:30.428659 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c7559c-52ea-4969-9079-8286cb8f22c1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:30.446504 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 112/231] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 15:00:30 2022] PUT /identity/v3/projects/65801518f2424895ab7af78ae7acb341/users/e22d1b017e4744cda46d80abfdbb947c/roles/ad9498a5e2064a468acc151fb2bce49e => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:00:30.506142 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07633ad8-a612-44fe-b0be-66976daffb65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:30.507699 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07633ad8-a612-44fe-b0be-66976daffb65 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:30.619956 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 120/232] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:00:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 15:00:30.625812 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c7559c-52ea-4969-9079-8286cb8f22c1 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:30.734404 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c7559c-52ea-4969-9079-8286cb8f22c1 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:30.770686 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b058b2-0251-4572-99f1-a9b9688a118f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:30.771594 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b058b2-0251-4572-99f1-a9b9688a118f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:30.771879 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b058b2-0251-4572-99f1-a9b9688a118f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:30.772156 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b058b2-0251-4572-99f1-a9b9688a118f None admin] PATH_INFO: `/v3/projects/65801518f2424895ab7af78ae7acb341` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:30.773359 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b058b2-0251-4572-99f1-a9b9688a118f None admin] RBAC: Authorizing `identity:get_project(project_id=65801518f2424895ab7af78ae7acb341)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:30.774822 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b058b2-0251-4572-99f1-a9b9688a118f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:30.776916 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 113/233] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:00:30 2022] GET /identity/v3/projects/65801518f2424895ab7af78ae7acb341 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:30.782758 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07633ad8-a612-44fe-b0be-66976daffb65 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:30.812103 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:30.894866 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07633ad8-a612-44fe-b0be-66976daffb65 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:30.944958 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f777395-144f-4829-943e-8799b691c248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:30.945683 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f777395-144f-4829-943e-8799b691c248 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:30.945976 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f777395-144f-4829-943e-8799b691c248 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:30.946258 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f777395-144f-4829-943e-8799b691c248 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:30.947120 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f777395-144f-4829-943e-8799b691c248 None admin] RBAC: Adding query filter params (role.id=8442c42e847a4b38b76f3a7fbe9ada85, scope.project.id=65801518f2424895ab7af78ae7acb341, user.id=dc9d9a4819c74e949d95bae75a1084ed) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:30.947433 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f777395-144f-4829-943e-8799b691c248 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:30.948920 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f777395-144f-4829-943e-8799b691c248 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:30.958176 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 121/234] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:00:30 2022] GET /identity/v3/role_assignments?user.id=dc9d9a4819c74e949d95bae75a1084ed&scope.project.id=65801518f2424895ab7af78ae7acb341&role.id=8442c42e847a4b38b76f3a7fbe9ada85 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:00:32.286232 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3e283-cffb-4421-8a50-6c71b84cd5df None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:32.286734 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3e283-cffb-4421-8a50-6c71b84cd5df None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:32.286810 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3e283-cffb-4421-8a50-6c71b84cd5df None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:32.288427 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 114/235] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:32.293750 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f052da-b59b-472a-91f2-61203df46d8f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:32.294127 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f052da-b59b-472a-91f2-61203df46d8f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:32.294303 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f052da-b59b-472a-91f2-61203df46d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:32.315852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b9f052da-b59b-472a-91f2-61203df46d8f None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:32.317058 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b9f052da-b59b-472a-91f2-61203df46d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:32.408650 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 122/236] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 15:00:32.443513 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f3e283-cffb-4421-8a50-6c71b84cd5df None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:32.478936 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:32.596548 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f3e283-cffb-4421-8a50-6c71b84cd5df None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:32.646371 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a393e5f-0a5b-4c2a-bf9a-fe5326d20bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:32.647604 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a393e5f-0a5b-4c2a-bf9a-fe5326d20bdb None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:32.648088 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a393e5f-0a5b-4c2a-bf9a-fe5326d20bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:32.648555 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a393e5f-0a5b-4c2a-bf9a-fe5326d20bdb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:32.664335 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a393e5f-0a5b-4c2a-bf9a-fe5326d20bdb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:32.672289 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-3a393e5f-0a5b-4c2a-bf9a-fe5326d20bdb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:00:32.672289 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:32.672289 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:32.672289 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:32.672289 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:32.672289 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:32.672289 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:32.672289 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:32.672289 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:32.672289 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:32.672289 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:32.672289 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:32.672289 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:32.672289 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:32.672289 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:32.672289 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:32.673674 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:32.673674 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:32.673674 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:32.673674 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:32.673674 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:32.673674 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:32.673674 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:32.673674 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:32.673674 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:32.673674 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:32.673674 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:32.673674 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:00:32.673674 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:32.673674 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:32.673674 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:32.673674 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:32.674483 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:32.674483 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:32.674483 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:32.674483 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:32.674483 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:32.674483 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:32.674483 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:32.674483 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:32.674483 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:32.674483 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:32.674483 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:00:32.674483 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:00:32.681381 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 115/237] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 15:00:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:00:32.689882 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f052da-b59b-472a-91f2-61203df46d8f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:32.728746 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:32.838947 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f052da-b59b-472a-91f2-61203df46d8f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:32.894963 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9b7874-2e1c-4c7c-8797-27c27b364d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:32.895307 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b7874-2e1c-4c7c-8797-27c27b364d09 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:32.895536 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b7874-2e1c-4c7c-8797-27c27b364d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:32.895820 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b7874-2e1c-4c7c-8797-27c27b364d09 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:32.896579 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9b7874-2e1c-4c7c-8797-27c27b364d09 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:32.896883 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9b7874-2e1c-4c7c-8797-27c27b364d09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:32.898271 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9b7874-2e1c-4c7c-8797-27c27b364d09 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:32.912591 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 123/238] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:00:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:32.918925 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a393e5f-0a5b-4c2a-bf9a-fe5326d20bdb None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:33.040146 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:33.057276 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a393e5f-0a5b-4c2a-bf9a-fe5326d20bdb None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:33.101099 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08af10af-04e2-4a60-82b7-1f10ed511d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:33.101986 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af10af-04e2-4a60-82b7-1f10ed511d3e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:33.102313 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af10af-04e2-4a60-82b7-1f10ed511d3e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:33.102591 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08af10af-04e2-4a60-82b7-1f10ed511d3e None admin] PATH_INFO: `/v3/users/e22d1b017e4744cda46d80abfdbb947c` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:33.107627 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08af10af-04e2-4a60-82b7-1f10ed511d3e None admin] RBAC: Authorizing `identity:get_user(user_id=e22d1b017e4744cda46d80abfdbb947c)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:33.109115 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08af10af-04e2-4a60-82b7-1f10ed511d3e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:33.111739 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 116/239] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:00:32 2022] GET /identity/v3/users/e22d1b017e4744cda46d80abfdbb947c => generated 283 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:33.117027 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9b7874-2e1c-4c7c-8797-27c27b364d09 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:33.198728 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6443afe1-0f15-4ea0-95c1-3e61815d6f0b None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:33.199071 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6443afe1-0f15-4ea0-95c1-3e61815d6f0b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:33.199243 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6443afe1-0f15-4ea0-95c1-3e61815d6f0b None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:33.200907 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 117/240] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:33.206865 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e872fa4a-6ab2-4c8d-9840-07433ea00cbc None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:33.207154 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e872fa4a-6ab2-4c8d-9840-07433ea00cbc None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:33.207430 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e872fa4a-6ab2-4c8d-9840-07433ea00cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:33.239904 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e872fa4a-6ab2-4c8d-9840-07433ea00cbc None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:33.241100 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e872fa4a-6ab2-4c8d-9840-07433ea00cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:33.255205 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:33.283200 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9b7874-2e1c-4c7c-8797-27c27b364d09 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:33.323746 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610c94e9-6eeb-46fe-ae61-a83e98059d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:33.324411 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610c94e9-6eeb-46fe-ae61-a83e98059d39 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:33.324683 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610c94e9-6eeb-46fe-ae61-a83e98059d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:33.324956 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610c94e9-6eeb-46fe-ae61-a83e98059d39 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:33.339721 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 118/241] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 15:00:33.341126 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370a0af-81ed-4811-865f-bae3db885e1f None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:33.341421 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370a0af-81ed-4811-865f-bae3db885e1f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:33.341699 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e370a0af-81ed-4811-865f-bae3db885e1f None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:33.347047 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 119/242] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:33.347047 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a7c35-9458-433d-abfe-952d4747046f None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:33.347047 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a7c35-9458-433d-abfe-952d4747046f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:33.347047 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a7c35-9458-433d-abfe-952d4747046f None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:33.350758 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 120/243] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:33 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:33.351975 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525dc3d-5f48-44c7-86e6-b958f3efd848 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:33.352282 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525dc3d-5f48-44c7-86e6-b958f3efd848 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:33.352555 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525dc3d-5f48-44c7-86e6-b958f3efd848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:33.369043 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4525dc3d-5f48-44c7-86e6-b958f3efd848 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:33.370265 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4525dc3d-5f48-44c7-86e6-b958f3efd848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:33.408864 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610c94e9-6eeb-46fe-ae61-a83e98059d39 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:33.410668 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610c94e9-6eeb-46fe-ae61-a83e98059d39 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:33.438905 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 121/244] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:00:33.438905 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5617db-9b6d-47f8-8966-6fab49eb42bc None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:33.438905 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5617db-9b6d-47f8-8966-6fab49eb42bc None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:33.439245 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5617db-9b6d-47f8-8966-6fab49eb42bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:33.461851 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-6b5617db-9b6d-47f8-8966-6fab49eb42bc None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:33.463521 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6b5617db-9b6d-47f8-8966-6fab49eb42bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:33.475047 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:33.506932 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:33.529538 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 124/245] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:00:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 15:00:33.530353 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610c94e9-6eeb-46fe-ae61-a83e98059d39 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:33.540139 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 122/246] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:00:33.547866 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5617db-9b6d-47f8-8966-6fab49eb42bc None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:33.640974 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610c94e9-6eeb-46fe-ae61-a83e98059d39 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:33.660990 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5617db-9b6d-47f8-8966-6fab49eb42bc None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:33.677380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a54e11-ec52-4215-b369-dbb66747931a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:33.678206 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a54e11-ec52-4215-b369-dbb66747931a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:33.678496 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a54e11-ec52-4215-b369-dbb66747931a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:33.678778 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a54e11-ec52-4215-b369-dbb66747931a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:33.680094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a54e11-ec52-4215-b369-dbb66747931a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:33.681577 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a54e11-ec52-4215-b369-dbb66747931a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:33.683343 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 125/247] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:00:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:33.687954 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a54e11-ec52-4215-b369-dbb66747931a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:33.704272 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb7fe9d-c036-4fc8-9f2b-34214f6b70c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:33.705262 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb7fe9d-c036-4fc8-9f2b-34214f6b70c9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:33.705619 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb7fe9d-c036-4fc8-9f2b-34214f6b70c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:33.705897 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb7fe9d-c036-4fc8-9f2b-34214f6b70c9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:33.717611 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb7fe9d-c036-4fc8-9f2b-34214f6b70c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:33.725467 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-ecb7fe9d-c036-4fc8-9f2b-34214f6b70c9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 15:00:33.725467 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:33.725467 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:33.725467 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:33.725467 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:33.725467 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:33.725467 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:33.725467 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:33.725467 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:33.725467 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:33.725467 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:33.725467 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:33.725467 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:33.725467 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:33.725467 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:33.725467 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:33.726469 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:33.726469 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:33.726469 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:33.726469 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:33.726469 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:33.726469 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:33.726469 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:33.726469 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:33.726469 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:33.726469 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:33.726469 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:33.726469 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:00:33.726469 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:33.726469 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:33.726469 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:33.726469 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:33.727319 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:33.727319 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:33.727319 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:33.727319 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:33.727319 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:33.727319 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:33.727319 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:33.727319 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:33.727319 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:33.727319 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:33.727319 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 15:00:33.727319 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:00:33.727319 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 123/248] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:00:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:00:33.727977 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb7fe9d-c036-4fc8-9f2b-34214f6b70c9 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:33.736265 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:33.752699 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:33.805652 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a54e11-ec52-4215-b369-dbb66747931a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:33.840744 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb7fe9d-c036-4fc8-9f2b-34214f6b70c9 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:33.844130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d03246-c42f-4aa1-bf16-c18dee8c8b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:33.845277 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d03246-c42f-4aa1-bf16-c18dee8c8b97 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:33.845563 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d03246-c42f-4aa1-bf16-c18dee8c8b97 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:33.845838 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d03246-c42f-4aa1-bf16-c18dee8c8b97 None admin] PATH_INFO: `/v3/projects/65801518f2424895ab7af78ae7acb341` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:33.847105 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d03246-c42f-4aa1-bf16-c18dee8c8b97 None admin] RBAC: Authorizing `identity:get_project(project_id=65801518f2424895ab7af78ae7acb341)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:33.848871 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d03246-c42f-4aa1-bf16-c18dee8c8b97 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:33.854966 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 126/249] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:00:33 2022] GET /identity/v3/projects/65801518f2424895ab7af78ae7acb341 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:33.854966 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d03246-c42f-4aa1-bf16-c18dee8c8b97 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:33.882763 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cd1ea0-574b-4009-9ab8-b71418ab4802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:33.883607 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd1ea0-574b-4009-9ab8-b71418ab4802 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:33.883892 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd1ea0-574b-4009-9ab8-b71418ab4802 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:33.884170 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd1ea0-574b-4009-9ab8-b71418ab4802 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:33.895010 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cd1ea0-574b-4009-9ab8-b71418ab4802 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:33.903048 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-f3cd1ea0-574b-4009-9ab8-b71418ab4802 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 15:00:33.903048 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:33.903048 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:33.903048 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:33.903048 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:33.903048 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:33.903048 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:33.903048 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:33.903048 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:33.903048 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:33.903048 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:33.903048 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:33.903048 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:33.903048 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:33.903048 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:33.903048 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:33.903928 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:33.903928 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:33.903928 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:33.903928 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:33.903928 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:33.903928 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:33.903928 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:33.903928 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:33.903928 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:33.903928 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:33.903928 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:33.903928 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:00:33.903928 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:33.903928 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:33.903928 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:33.903928 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:33.904723 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:33.904723 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:33.904723 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:33.904723 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:33.904723 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:33.904723 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:33.904723 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:33.904723 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:33.904723 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:33.904723 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:33.904723 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 15:00:33.904723 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:00:33.904723 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 124/250] 10.222.0.50 () {56 vars in 1114 bytes} [Fri May 27 15:00:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:00:33.905373 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cd1ea0-574b-4009-9ab8-b71418ab4802 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:33.953125 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d03246-c42f-4aa1-bf16-c18dee8c8b97 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:33.966342 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:33.971464 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:33.988675 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11315931-be5d-4cfe-bc4e-eea99112d7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:33.989333 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11315931-be5d-4cfe-bc4e-eea99112d7f1 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:33.989616 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11315931-be5d-4cfe-bc4e-eea99112d7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:33.989887 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11315931-be5d-4cfe-bc4e-eea99112d7f1 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:33.990576 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11315931-be5d-4cfe-bc4e-eea99112d7f1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:33.992049 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11315931-be5d-4cfe-bc4e-eea99112d7f1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:34.006242 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cd1ea0-574b-4009-9ab8-b71418ab4802 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:34.012110 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 127/251] 10.222.0.50 () {60 vars in 1132 bytes} [Fri May 27 15:00:33 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:00:34.012933 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11315931-be5d-4cfe-bc4e-eea99112d7f1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:34.064991 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cceae11-0120-4617-94b6-201879f318f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:34.065672 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cceae11-0120-4617-94b6-201879f318f1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:34.065959 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cceae11-0120-4617-94b6-201879f318f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:34.066240 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cceae11-0120-4617-94b6-201879f318f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:34.067040 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cceae11-0120-4617-94b6-201879f318f1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:34.067353 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cceae11-0120-4617-94b6-201879f318f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:34.068833 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cceae11-0120-4617-94b6-201879f318f1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:34.078458 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 125/252] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 15:00:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:34.085203 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828735f2-5a63-4259-b656-545195bdf697 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:34.085509 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828735f2-5a63-4259-b656-545195bdf697 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:34.085775 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828735f2-5a63-4259-b656-545195bdf697 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:34.087835 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 126/253] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:34.088502 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828735f2-5a63-4259-b656-545195bdf697 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:34.129854 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11315931-be5d-4cfe-bc4e-eea99112d7f1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:34.180229 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044bdd7d-952c-4615-9f7b-665b76e615ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:34.180906 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044bdd7d-952c-4615-9f7b-665b76e615ed None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:34.181191 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044bdd7d-952c-4615-9f7b-665b76e615ed None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:34.181474 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044bdd7d-952c-4615-9f7b-665b76e615ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:34.182314 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044bdd7d-952c-4615-9f7b-665b76e615ed None admin] RBAC: Adding query filter params (role.id=759d845eb6284e42931b4265af448ebc, scope.project.id=65801518f2424895ab7af78ae7acb341, user.id=e22d1b017e4744cda46d80abfdbb947c) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:34.182963 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044bdd7d-952c-4615-9f7b-665b76e615ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:34.184164 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044bdd7d-952c-4615-9f7b-665b76e615ed None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:34.191940 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 128/254] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:00:34 2022] GET /identity/v3/role_assignments?user.id=e22d1b017e4744cda46d80abfdbb947c&scope.project.id=65801518f2424895ab7af78ae7acb341&role.id=759d845eb6284e42931b4265af448ebc => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:34.192684 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044bdd7d-952c-4615-9f7b-665b76e615ed None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:34.212138 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828735f2-5a63-4259-b656-545195bdf697 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:34.244512 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:34.253305 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:34.275063 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc510e5c-5b99-4fcf-97af-3854d3d98e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:34.276122 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc510e5c-5b99-4fcf-97af-3854d3d98e59 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:34.276852 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc510e5c-5b99-4fcf-97af-3854d3d98e59 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:34.277313 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc510e5c-5b99-4fcf-97af-3854d3d98e59 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:34.278091 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc510e5c-5b99-4fcf-97af-3854d3d98e59 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:34.278871 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc510e5c-5b99-4fcf-97af-3854d3d98e59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:34.280328 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc510e5c-5b99-4fcf-97af-3854d3d98e59 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:34.296947 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 127/255] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:00:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:34.303452 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a51387-e831-4aec-9588-159207b6ffd8 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:34.304409 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a51387-e831-4aec-9588-159207b6ffd8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:34.304699 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a51387-e831-4aec-9588-159207b6ffd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:34.324417 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e0a51387-e831-4aec-9588-159207b6ffd8 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:34.325641 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e0a51387-e831-4aec-9588-159207b6ffd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:34.340869 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044bdd7d-952c-4615-9f7b-665b76e615ed None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:34.390885 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cf687c-c188-48a8-9aec-84fb46b5e690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:34.392332 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cf687c-c188-48a8-9aec-84fb46b5e690 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:34.392611 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cf687c-c188-48a8-9aec-84fb46b5e690 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:34.392880 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cf687c-c188-48a8-9aec-84fb46b5e690 None admin] PATH_INFO: `/v3/users/8ee4b55c12aa4a35904984a26a6da877` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:34.412214 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cf687c-c188-48a8-9aec-84fb46b5e690 None admin] RBAC: Authorizing `identity:get_user(user_id=8ee4b55c12aa4a35904984a26a6da877)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:34.413784 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cf687c-c188-48a8-9aec-84fb46b5e690 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:34.416186 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 129/256] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:00:34 2022] GET /identity/v3/users/8ee4b55c12aa4a35904984a26a6da877 => generated 297 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:34.417245 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cf687c-c188-48a8-9aec-84fb46b5e690 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:34.430311 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 128/257] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:00:34.432005 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a51387-e831-4aec-9588-159207b6ffd8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:34.547008 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:34.555266 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cf687c-c188-48a8-9aec-84fb46b5e690 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:34.566288 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a51387-e831-4aec-9588-159207b6ffd8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:34.589129 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:34.606593 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19571193-dae6-415d-80e3-7406e78567d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:34.607531 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19571193-dae6-415d-80e3-7406e78567d0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:34.607815 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19571193-dae6-415d-80e3-7406e78567d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:34.608093 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19571193-dae6-415d-80e3-7406e78567d0 None admin] PATH_INFO: `/v3/users/e22d1b017e4744cda46d80abfdbb947c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:34.609309 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19571193-dae6-415d-80e3-7406e78567d0 None admin] RBAC: Authorizing `identity:get_user(user_id=e22d1b017e4744cda46d80abfdbb947c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:34.611570 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19571193-dae6-415d-80e3-7406e78567d0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:34.612815 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383d97c1-ea1f-41d4-a85e-2a9fc2d98af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:34.613234 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 130/258] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:00:34 2022] GET /identity/v3/users/e22d1b017e4744cda46d80abfdbb947c => generated 283 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:34.613904 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d97c1-ea1f-41d4-a85e-2a9fc2d98af0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:34.614032 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19571193-dae6-415d-80e3-7406e78567d0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:34.614621 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d97c1-ea1f-41d4-a85e-2a9fc2d98af0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:34.615207 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383d97c1-ea1f-41d4-a85e-2a9fc2d98af0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:34.696922 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383d97c1-ea1f-41d4-a85e-2a9fc2d98af0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:34.698793 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383d97c1-ea1f-41d4-a85e-2a9fc2d98af0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:34.752690 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19571193-dae6-415d-80e3-7406e78567d0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:34.798017 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509a1c57-b945-444f-bb4d-a07c8fb67ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:34.799629 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a1c57-b945-444f-bb4d-a07c8fb67ea8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:34.800398 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a1c57-b945-444f-bb4d-a07c8fb67ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:34.801165 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509a1c57-b945-444f-bb4d-a07c8fb67ea8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:34.816194 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509a1c57-b945-444f-bb4d-a07c8fb67ea8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:34.821891 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 129/259] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:00:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 15:00:34.825930 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-509a1c57-b945-444f-bb4d-a07c8fb67ea8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 15:00:34.825930 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:34.825930 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:34.825930 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:34.825930 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:34.825930 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:34.825930 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:34.825930 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:34.825930 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:34.825930 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:34.825930 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:34.825930 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:34.825930 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:34.825930 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:34.825930 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:34.825930 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:34.826802 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:34.826802 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:34.826802 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:34.826802 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:34.826802 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:34.826802 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:34.826802 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:34.826802 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:34.826802 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:34.826802 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:34.826802 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:34.826802 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:00:34.826802 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:34.826802 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:34.826802 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:34.826802 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:34.827701 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:34.827701 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:34.827701 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:34.827701 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:34.827701 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:34.827701 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:34.827701 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:34.827701 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:34.827701 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:34.827701 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:34.827701 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 15:00:34.827701 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:00:34.828298 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383d97c1-ea1f-41d4-a85e-2a9fc2d98af0 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:34.830896 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 131/260] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:00:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:00:34.832737 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509a1c57-b945-444f-bb4d-a07c8fb67ea8 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:34.855223 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:34.866129 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:34.938065 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383d97c1-ea1f-41d4-a85e-2a9fc2d98af0 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:34.972970 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509a1c57-b945-444f-bb4d-a07c8fb67ea8 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:34.975353 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf5015c-807b-4975-be8b-a2f3f7e3caf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:34.976107 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5015c-807b-4975-be8b-a2f3f7e3caf8 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:34.976463 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5015c-807b-4975-be8b-a2f3f7e3caf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:34.977465 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5015c-807b-4975-be8b-a2f3f7e3caf8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:35.017572 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb4ee9-dbd2-4923-b631-21b4033a8894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:35.019199 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb4ee9-dbd2-4923-b631-21b4033a8894 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:35.019687 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb4ee9-dbd2-4923-b631-21b4033a8894 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:35.020133 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bb4ee9-dbd2-4923-b631-21b4033a8894 None admin] PATH_INFO: `/v3/projects/65801518f2424895ab7af78ae7acb341` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:35.022242 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bb4ee9-dbd2-4923-b631-21b4033a8894 None admin] RBAC: Authorizing `identity:get_project(project_id=65801518f2424895ab7af78ae7acb341)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:35.024182 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bb4ee9-dbd2-4923-b631-21b4033a8894 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:35.031248 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 132/261] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:00:34 2022] GET /identity/v3/projects/65801518f2424895ab7af78ae7acb341 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:35.033136 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb4ee9-dbd2-4923-b631-21b4033a8894 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:35.053207 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf5015c-807b-4975-be8b-a2f3f7e3caf8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:35.055441 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf5015c-807b-4975-be8b-a2f3f7e3caf8 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:35.108863 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:35.135203 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:35.154253 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bb4ee9-dbd2-4923-b631-21b4033a8894 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:35.190256 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 130/262] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:00:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 15:00:35.191456 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf5015c-807b-4975-be8b-a2f3f7e3caf8 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:35.197479 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243ae507-6024-4696-9091-82c09494f767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:35.198155 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ae507-6024-4696-9091-82c09494f767 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:35.198440 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ae507-6024-4696-9091-82c09494f767 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:35.198718 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ae507-6024-4696-9091-82c09494f767 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:35.199591 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243ae507-6024-4696-9091-82c09494f767 None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:35.199902 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243ae507-6024-4696-9091-82c09494f767 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:35.201278 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243ae507-6024-4696-9091-82c09494f767 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:35.211167 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 133/263] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 15:00:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:35.211774 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243ae507-6024-4696-9091-82c09494f767 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:35.317677 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243ae507-6024-4696-9091-82c09494f767 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:35.322203 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf5015c-807b-4975-be8b-a2f3f7e3caf8 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:35.345616 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:35.355562 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd25d0a-10d1-44c9-881f-33e20d24f194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:35.356272 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd25d0a-10d1-44c9-881f-33e20d24f194 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:35.356559 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd25d0a-10d1-44c9-881f-33e20d24f194 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:35.356832 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd25d0a-10d1-44c9-881f-33e20d24f194 None admin] PATH_INFO: `/v3/projects/65801518f2424895ab7af78ae7acb341` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:35.358008 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd25d0a-10d1-44c9-881f-33e20d24f194 None admin] RBAC: Authorizing `identity:get_project(project_id=65801518f2424895ab7af78ae7acb341)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:35.359588 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd25d0a-10d1-44c9-881f-33e20d24f194 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:35.361698 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 134/264] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:00:35 2022] GET /identity/v3/projects/65801518f2424895ab7af78ae7acb341 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:35.362382 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd25d0a-10d1-44c9-881f-33e20d24f194 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:35.373885 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6b35c-d60e-41ac-b5b9-6a97372345a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:35.375959 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6b35c-d60e-41ac-b5b9-6a97372345a7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:35.377147 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6b35c-d60e-41ac-b5b9-6a97372345a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:35.377958 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f6b35c-d60e-41ac-b5b9-6a97372345a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:35.379084 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f6b35c-d60e-41ac-b5b9-6a97372345a7 None admin] RBAC: Adding query filter params (role.id=1b313eb2327b44b7a11d92914db65bb9, scope.project.id=65801518f2424895ab7af78ae7acb341, user.id=8ee4b55c12aa4a35904984a26a6da877) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:35.379910 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f6b35c-d60e-41ac-b5b9-6a97372345a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:35.382574 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f6b35c-d60e-41ac-b5b9-6a97372345a7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:35.390942 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:35.393284 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 131/265] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:00:35 2022] GET /identity/v3/role_assignments?user.id=8ee4b55c12aa4a35904984a26a6da877&scope.project.id=65801518f2424895ab7af78ae7acb341&role.id=1b313eb2327b44b7a11d92914db65bb9 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:35.394475 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6b35c-d60e-41ac-b5b9-6a97372345a7 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:35.542882 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd25d0a-10d1-44c9-881f-33e20d24f194 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:35.566190 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f6b35c-d60e-41ac-b5b9-6a97372345a7 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:35.587362 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e4e38c-9bbe-43a8-bffe-02e789456dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:35.589101 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4e38c-9bbe-43a8-bffe-02e789456dd0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:35.589570 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4e38c-9bbe-43a8-bffe-02e789456dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:35.590019 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4e38c-9bbe-43a8-bffe-02e789456dd0 None admin] PATH_INFO: `/v3/users/dc9d9a4819c74e949d95bae75a1084ed` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:35.591507 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e4e38c-9bbe-43a8-bffe-02e789456dd0 None admin] RBAC: Authorizing `identity:get_user(user_id=dc9d9a4819c74e949d95bae75a1084ed)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:35.593778 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e4e38c-9bbe-43a8-bffe-02e789456dd0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:35.596518 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 135/266] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:00:35 2022] GET /identity/v3/users/dc9d9a4819c74e949d95bae75a1084ed => generated 297 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:35.601897 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e4e38c-9bbe-43a8-bffe-02e789456dd0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:35.628619 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c0ed03-bbb4-4e76-a29b-79754d06e112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:35.629293 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c0ed03-bbb4-4e76-a29b-79754d06e112 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:35.629582 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c0ed03-bbb4-4e76-a29b-79754d06e112 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:35.629863 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c0ed03-bbb4-4e76-a29b-79754d06e112 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:35.630693 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c0ed03-bbb4-4e76-a29b-79754d06e112 None admin] RBAC: Adding query filter params (role.id=ad9498a5e2064a468acc151fb2bce49e, scope.project.id=65801518f2424895ab7af78ae7acb341, user.id=e22d1b017e4744cda46d80abfdbb947c) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:35.631421 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c0ed03-bbb4-4e76-a29b-79754d06e112 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:35.632486 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c0ed03-bbb4-4e76-a29b-79754d06e112 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:35.647977 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 132/267] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:00:35 2022] GET /identity/v3/role_assignments?user.id=e22d1b017e4744cda46d80abfdbb947c&scope.project.id=65801518f2424895ab7af78ae7acb341&role.id=ad9498a5e2064a468acc151fb2bce49e => generated 608 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:35.711674 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:35.723850 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e4e38c-9bbe-43a8-bffe-02e789456dd0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:35.774789 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2768e5-c773-4c36-8754-315d8ebd2a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:35.776016 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2768e5-c773-4c36-8754-315d8ebd2a65 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:35.776500 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2768e5-c773-4c36-8754-315d8ebd2a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:35.776957 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2768e5-c773-4c36-8754-315d8ebd2a65 None admin] PATH_INFO: `/v3/projects/65801518f2424895ab7af78ae7acb341` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:35.778405 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2768e5-c773-4c36-8754-315d8ebd2a65 None admin] RBAC: Authorizing `identity:get_project(project_id=65801518f2424895ab7af78ae7acb341)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:35.780089 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2768e5-c773-4c36-8754-315d8ebd2a65 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:35.782669 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 136/268] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:00:35 2022] GET /identity/v3/projects/65801518f2424895ab7af78ae7acb341 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:00:35.788553 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c0ed03-bbb4-4e76-a29b-79754d06e112 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:35.864912 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:35.920178 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c0ed03-bbb4-4e76-a29b-79754d06e112 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:35.958054 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25e6f87-01f4-4221-91ff-eb2d466d74b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:35.958983 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e6f87-01f4-4221-91ff-eb2d466d74b7 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:35.959274 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e6f87-01f4-4221-91ff-eb2d466d74b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:35.959552 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e6f87-01f4-4221-91ff-eb2d466d74b7 None admin] PATH_INFO: `/v3/projects/65801518f2424895ab7af78ae7acb341/users/dc9d9a4819c74e949d95bae75a1084ed/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:35.961563 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25e6f87-01f4-4221-91ff-eb2d466d74b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=65801518f2424895ab7af78ae7acb341, user_id=dc9d9a4819c74e949d95bae75a1084ed, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:35.963318 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25e6f87-01f4-4221-91ff-eb2d466d74b7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:35.984856 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 133/269] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 15:00:35 2022] PUT /identity/v3/projects/65801518f2424895ab7af78ae7acb341/users/dc9d9a4819c74e949d95bae75a1084ed/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:00:36.656736 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d213bed-0515-41ee-a370-9b5ce1549bce None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:36.656736 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d213bed-0515-41ee-a370-9b5ce1549bce None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:36.656736 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d213bed-0515-41ee-a370-9b5ce1549bce None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:36.657732 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 137/270] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:36.663143 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b064c-e8ab-47b4-bd95-fb63b6f9d23b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:36.663928 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b064c-e8ab-47b4-bd95-fb63b6f9d23b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:36.664419 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b064c-e8ab-47b4-bd95-fb63b6f9d23b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:36.681407 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e25b064c-e8ab-47b4-bd95-fb63b6f9d23b None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:36.682936 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e25b064c-e8ab-47b4-bd95-fb63b6f9d23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:36.686293 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf60829-5dab-4c37-b783-3a3a3440a650 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:36.686587 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf60829-5dab-4c37-b783-3a3a3440a650 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:36.686893 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf60829-5dab-4c37-b783-3a3a3440a650 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:36.689040 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 138/271] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 15:00:36.694066 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ab6e8-9077-4191-95ea-8fdf8bfda619 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:36.694352 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ab6e8-9077-4191-95ea-8fdf8bfda619 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:36.694632 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ab6e8-9077-4191-95ea-8fdf8bfda619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:36.712500 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7c6ab6e8-9077-4191-95ea-8fdf8bfda619 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:36.713938 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7c6ab6e8-9077-4191-95ea-8fdf8bfda619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:36.764533 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 134/272] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:00:36.789351 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25b064c-e8ab-47b4-bd95-fb63b6f9d23b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:36.800289 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 139/273] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 15:00:36.824304 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ab6e8-9077-4191-95ea-8fdf8bfda619 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:36.826355 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:36.911332 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25b064c-e8ab-47b4-bd95-fb63b6f9d23b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:36.926557 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:36.941244 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ab6e8-9077-4191-95ea-8fdf8bfda619 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:36.954383 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f12e3e-1b4c-4108-a1de-dbdf01b8c58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:36.955215 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f12e3e-1b4c-4108-a1de-dbdf01b8c58e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:36.955499 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f12e3e-1b4c-4108-a1de-dbdf01b8c58e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:36.955772 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f12e3e-1b4c-4108-a1de-dbdf01b8c58e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:36.969000 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f12e3e-1b4c-4108-a1de-dbdf01b8c58e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:36.978304 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-f8f12e3e-1b4c-4108-a1de-dbdf01b8c58e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 15:00:36.978304 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:36.978304 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:36.978304 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:36.978304 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:36.978304 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:36.978304 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:36.978304 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:36.978304 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:36.978304 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:36.978304 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:36.978304 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:36.978304 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:36.978304 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:36.978304 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:36.978304 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:36.979250 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:36.979250 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:36.979250 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:36.979250 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:36.979250 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:36.979250 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:36.979250 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:36.979250 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:36.979250 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:36.979250 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:36.979250 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:36.979250 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:00:36.979250 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:36.979250 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:36.979250 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:36.979250 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:36.980053 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:36.980053 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:36.980053 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:36.980053 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:36.980053 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:36.980053 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:36.980053 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:36.980053 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:36.980053 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:36.980053 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:36.980053 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 15:00:36.980053 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:00:36.980053 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 135/274] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:00:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:00:36.987042 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f12e3e-1b4c-4108-a1de-dbdf01b8c58e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:36.988488 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0208235-b792-477d-b115-b9286251b8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:36.989519 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0208235-b792-477d-b115-b9286251b8a5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:36.990046 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0208235-b792-477d-b115-b9286251b8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:36.990516 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0208235-b792-477d-b115-b9286251b8a5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:36.992134 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0208235-b792-477d-b115-b9286251b8a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:36.994918 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0208235-b792-477d-b115-b9286251b8a5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:36.997057 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 140/275] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:00:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:00:37.003681 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0208235-b792-477d-b115-b9286251b8a5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:37.106303 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:37.115066 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f12e3e-1b4c-4108-a1de-dbdf01b8c58e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:37.128758 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0208235-b792-477d-b115-b9286251b8a5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:37.161841 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba91c153-c916-49d6-8034-a0f84d87bf89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:37.162915 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba91c153-c916-49d6-8034-a0f84d87bf89 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:37.163402 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba91c153-c916-49d6-8034-a0f84d87bf89 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:37.163893 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba91c153-c916-49d6-8034-a0f84d87bf89 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:37.164940 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba91c153-c916-49d6-8034-a0f84d87bf89 None admin] RBAC: Adding query filter params (name=member) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:37.165491 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba91c153-c916-49d6-8034-a0f84d87bf89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:37.167137 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba91c153-c916-49d6-8034-a0f84d87bf89 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:37.174088 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2a6174-1049-4daf-b462-2c691d6adfc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:37.174777 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a6174-1049-4daf-b462-2c691d6adfc5 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:37.175200 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a6174-1049-4daf-b462-2c691d6adfc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:37.175591 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2a6174-1049-4daf-b462-2c691d6adfc5 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:37.176369 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2a6174-1049-4daf-b462-2c691d6adfc5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:37.178906 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2a6174-1049-4daf-b462-2c691d6adfc5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:37.187440 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 136/276] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 15:00:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:37.191390 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba91c153-c916-49d6-8034-a0f84d87bf89 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:37.202303 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 141/277] 10.222.0.50 () {60 vars in 1132 bytes} [Fri May 27 15:00:37 2022] POST /identity/v3/users => generated 293 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:00:37.315203 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba91c153-c916-49d6-8034-a0f84d87bf89 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:37.354450 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8c76a-4ad9-4f8d-9fc6-8f132871cd06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:37.355727 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8c76a-4ad9-4f8d-9fc6-8f132871cd06 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:37.356216 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8c76a-4ad9-4f8d-9fc6-8f132871cd06 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:37.356715 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8c76a-4ad9-4f8d-9fc6-8f132871cd06 None admin] PATH_INFO: `/v3/users/6f52f8f1850247cfbbb016d62c60de86` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:37.373886 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f8c76a-4ad9-4f8d-9fc6-8f132871cd06 None admin] RBAC: Authorizing `identity:get_user(user_id=6f52f8f1850247cfbbb016d62c60de86)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:37.375959 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f8c76a-4ad9-4f8d-9fc6-8f132871cd06 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:37.378340 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 137/278] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:00:37 2022] GET /identity/v3/users/6f52f8f1850247cfbbb016d62c60de86 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:37.384447 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2a6174-1049-4daf-b462-2c691d6adfc5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:37.402372 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:37.495842 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2a6174-1049-4daf-b462-2c691d6adfc5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:37.533659 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221bb0bd-a7ef-4248-aed0-b1f2dd784279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:37.534662 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221bb0bd-a7ef-4248-aed0-b1f2dd784279 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:37.535232 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221bb0bd-a7ef-4248-aed0-b1f2dd784279 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:37.535716 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221bb0bd-a7ef-4248-aed0-b1f2dd784279 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:37.536759 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221bb0bd-a7ef-4248-aed0-b1f2dd784279 None admin] RBAC: Adding query filter params (role.id=8442c42e847a4b38b76f3a7fbe9ada85, user.id=6f52f8f1850247cfbbb016d62c60de86, scope.system=all) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:37.537267 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221bb0bd-a7ef-4248-aed0-b1f2dd784279 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:37.538982 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221bb0bd-a7ef-4248-aed0-b1f2dd784279 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:37.544805 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 142/279] 10.222.0.50 () {56 vars in 1308 bytes} [Fri May 27 15:00:37 2022] GET /identity/v3/role_assignments?user.id=6f52f8f1850247cfbbb016d62c60de86&scope.system=all&role.id=8442c42e847a4b38b76f3a7fbe9ada85 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:37.792458 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c67c08-0523-4ef0-a7d4-1bd9c7d05d4b None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:37.792984 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c67c08-0523-4ef0-a7d4-1bd9c7d05d4b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:37.793061 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c67c08-0523-4ef0-a7d4-1bd9c7d05d4b None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:37.794655 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 138/280] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:37.800789 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e601e82-e1d6-4af8-8b78-58572453f8a7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:37.801561 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e601e82-e1d6-4af8-8b78-58572453f8a7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:37.801838 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e601e82-e1d6-4af8-8b78-58572453f8a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:37.827918 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8e601e82-e1d6-4af8-8b78-58572453f8a7 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:37.829132 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8e601e82-e1d6-4af8-8b78-58572453f8a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:37.925172 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 143/281] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:00:37.966095 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c67c08-0523-4ef0-a7d4-1bd9c7d05d4b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:37.994753 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:38.111132 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c67c08-0523-4ef0-a7d4-1bd9c7d05d4b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:38.151408 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c308bbd-a4e6-4a05-a272-acdeb522eb62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:38.152232 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c308bbd-a4e6-4a05-a272-acdeb522eb62 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:38.152518 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c308bbd-a4e6-4a05-a272-acdeb522eb62 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:38.152796 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c308bbd-a4e6-4a05-a272-acdeb522eb62 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:38.170313 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c308bbd-a4e6-4a05-a272-acdeb522eb62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:38.179164 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-6c308bbd-a4e6-4a05-a272-acdeb522eb62 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 15:00:38.179164 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:38.179164 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:38.179164 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:38.179164 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:38.179164 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:38.179164 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:38.179164 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:38.179164 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:38.179164 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:38.179164 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:38.179164 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:38.179164 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:38.179164 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:38.179164 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:38.179164 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:38.180038 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:38.180038 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:38.180038 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:38.180038 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:38.180038 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:38.180038 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:38.180038 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:38.180038 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:38.180038 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:38.180038 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:38.180038 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:38.180038 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:00:38.180038 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:38.180038 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:38.180038 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:38.180038 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:38.180839 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:38.180839 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:38.180839 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:38.180839 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:38.180839 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:38.180839 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:38.180839 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:38.180839 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:38.180839 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:38.180839 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:38.180839 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 15:00:38.180839 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:00:38.185090 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 139/282] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:00:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:00:38.186441 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e601e82-e1d6-4af8-8b78-58572453f8a7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:38.198490 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:38.367947 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e5576c-9b7e-4a19-9d03-de2f4380a45d None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:38.367947 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e5576c-9b7e-4a19-9d03-de2f4380a45d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:38.368175 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e5576c-9b7e-4a19-9d03-de2f4380a45d None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:38.371005 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 140/283] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:38.374158 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e601e82-e1d6-4af8-8b78-58572453f8a7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:38.374956 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ee203-1372-4a80-be51-63a003db302f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:38.375243 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ee203-1372-4a80-be51-63a003db302f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:38.375559 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5ee203-1372-4a80-be51-63a003db302f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:38.415934 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8a5ee203-1372-4a80-be51-63a003db302f None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:38.417101 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8a5ee203-1372-4a80-be51-63a003db302f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:38.417460 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d10f8-65aa-45a2-b484-67890f9bb7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:38.418370 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d10f8-65aa-45a2-b484-67890f9bb7b1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:38.418961 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d10f8-65aa-45a2-b484-67890f9bb7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:38.419448 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d10f8-65aa-45a2-b484-67890f9bb7b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:38.420528 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58d10f8-65aa-45a2-b484-67890f9bb7b1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:38.421027 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58d10f8-65aa-45a2-b484-67890f9bb7b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:38.423477 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58d10f8-65aa-45a2-b484-67890f9bb7b1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:38.433139 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 144/284] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 15:00:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:38.438942 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d10f8-65aa-45a2-b484-67890f9bb7b1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:38.538891 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 141/285] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 15:00:38.552945 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5ee203-1372-4a80-be51-63a003db302f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:38.561361 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:38.572140 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:38.641804 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d10f8-65aa-45a2-b484-67890f9bb7b1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:38.663927 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5ee203-1372-4a80-be51-63a003db302f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:38.700830 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a3c2f-b6b3-408d-b217-78addceddf2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:38.701864 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a3c2f-b6b3-408d-b217-78addceddf2f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:38.702285 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a3c2f-b6b3-408d-b217-78addceddf2f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:38.702697 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a3c2f-b6b3-408d-b217-78addceddf2f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:38.709931 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056daf37-e425-440a-96bc-dfcd5182c589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:38.710945 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056daf37-e425-440a-96bc-dfcd5182c589 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:38.711232 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056daf37-e425-440a-96bc-dfcd5182c589 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:38.711713 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056daf37-e425-440a-96bc-dfcd5182c589 None admin] PATH_INFO: `/v3/users/8ee4b55c12aa4a35904984a26a6da877` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:38.713069 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056daf37-e425-440a-96bc-dfcd5182c589 None admin] RBAC: Authorizing `identity:get_user(user_id=8ee4b55c12aa4a35904984a26a6da877)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:38.714514 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056daf37-e425-440a-96bc-dfcd5182c589 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:38.715009 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1a3c2f-b6b3-408d-b217-78addceddf2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:38.723239 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-5a1a3c2f-b6b3-408d-b217-78addceddf2f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 15:00:38.723239 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:38.723239 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:38.723239 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:38.723239 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:38.723239 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:38.723239 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:38.723239 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:38.723239 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:38.723239 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:38.723239 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:38.723239 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:38.723239 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:38.723239 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:38.723239 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:38.723239 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:38.724284 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:38.724284 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:38.724284 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:38.724284 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:38.724284 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:38.724284 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:38.724284 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:38.724284 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:38.724284 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:38.724284 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:38.724284 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:38.724284 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:00:38.724284 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:38.724284 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:38.724284 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:38.724284 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:38.725207 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:38.725207 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:38.725207 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:38.725207 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:38.725207 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:38.725207 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:38.725207 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:38.725207 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:38.725207 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:38.725207 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:38.725207 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 15:00:38.725207 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:00:38.725801 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 145/286] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:00:38 2022] GET /identity/v3/users/8ee4b55c12aa4a35904984a26a6da877 => generated 297 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:38.732408 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 142/287] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:00:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:00:38.733245 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056daf37-e425-440a-96bc-dfcd5182c589 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:38.733826 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a3c2f-b6b3-408d-b217-78addceddf2f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:38.806334 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:38.846319 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a3c2f-b6b3-408d-b217-78addceddf2f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:38.890036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbad241e-477f-45fd-a678-e9da622d4aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:38.891047 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad241e-477f-45fd-a678-e9da622d4aef None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:38.891657 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad241e-477f-45fd-a678-e9da622d4aef None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:38.892265 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbad241e-477f-45fd-a678-e9da622d4aef None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:38.893205 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbad241e-477f-45fd-a678-e9da622d4aef None admin] RBAC: Adding query filter params (name=member) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:38.893651 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbad241e-477f-45fd-a678-e9da622d4aef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:38.895248 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbad241e-477f-45fd-a678-e9da622d4aef None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:38.915865 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 143/288] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 15:00:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:38.919394 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbad241e-477f-45fd-a678-e9da622d4aef None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:38.954890 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056daf37-e425-440a-96bc-dfcd5182c589 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:39.028277 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ccd31-9c92-407c-a6c8-17671190db9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:39.029374 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ccd31-9c92-407c-a6c8-17671190db9c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:39.029733 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ccd31-9c92-407c-a6c8-17671190db9c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:39.030128 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ccd31-9c92-407c-a6c8-17671190db9c None admin] PATH_INFO: `/v3/projects/65801518f2424895ab7af78ae7acb341` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:39.031500 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0ccd31-9c92-407c-a6c8-17671190db9c None admin] RBAC: Authorizing `identity:get_project(project_id=65801518f2424895ab7af78ae7acb341)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:39.033122 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0ccd31-9c92-407c-a6c8-17671190db9c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:39.035134 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 146/289] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:00:38 2022] GET /identity/v3/projects/65801518f2424895ab7af78ae7acb341 => generated 302 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:00:39.040742 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ccd31-9c92-407c-a6c8-17671190db9c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:39.050499 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:39.053956 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbad241e-477f-45fd-a678-e9da622d4aef None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:39.092633 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ace05e-3ac4-44b5-b74a-e352e0250cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:39.093870 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ace05e-3ac4-44b5-b74a-e352e0250cf4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:39.094434 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ace05e-3ac4-44b5-b74a-e352e0250cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:39.095145 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ace05e-3ac4-44b5-b74a-e352e0250cf4 None admin] PATH_INFO: `/v3/users/dc9d9a4819c74e949d95bae75a1084ed` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:39.096239 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:39.103133 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ace05e-3ac4-44b5-b74a-e352e0250cf4 None admin] RBAC: Authorizing `identity:get_user(user_id=dc9d9a4819c74e949d95bae75a1084ed)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:39.106699 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ace05e-3ac4-44b5-b74a-e352e0250cf4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:39.106699 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 144/290] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:00:38 2022] GET /identity/v3/users/dc9d9a4819c74e949d95bae75a1084ed => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:00:39.115781 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ace05e-3ac4-44b5-b74a-e352e0250cf4 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:39.172898 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ccd31-9c92-407c-a6c8-17671190db9c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:39.209452 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b735d8c-b376-4750-9f11-4fae1ce97e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:39.210649 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b735d8c-b376-4750-9f11-4fae1ce97e13 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:39.211108 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b735d8c-b376-4750-9f11-4fae1ce97e13 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:39.211519 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b735d8c-b376-4750-9f11-4fae1ce97e13 None admin] PATH_INFO: `/v3/projects/65801518f2424895ab7af78ae7acb341/users/8ee4b55c12aa4a35904984a26a6da877/roles/1b313eb2327b44b7a11d92914db65bb9` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:39.213878 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b735d8c-b376-4750-9f11-4fae1ce97e13 None admin] RBAC: Authorizing `identity:create_grant(project_id=65801518f2424895ab7af78ae7acb341, user_id=8ee4b55c12aa4a35904984a26a6da877, role_id=1b313eb2327b44b7a11d92914db65bb9)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:39.215509 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b735d8c-b376-4750-9f11-4fae1ce97e13 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:39.238696 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 147/291] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 15:00:39 2022] PUT /identity/v3/projects/65801518f2424895ab7af78ae7acb341/users/8ee4b55c12aa4a35904984a26a6da877/roles/1b313eb2327b44b7a11d92914db65bb9 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:00:39.306016 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ace05e-3ac4-44b5-b74a-e352e0250cf4 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:39.357466 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99da77e-1381-4740-a308-f86119f3bcd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:39.358372 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99da77e-1381-4740-a308-f86119f3bcd3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:39.358827 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99da77e-1381-4740-a308-f86119f3bcd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:39.359266 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99da77e-1381-4740-a308-f86119f3bcd3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:39.442006 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:39.450580 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99da77e-1381-4740-a308-f86119f3bcd3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:39.456345 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99da77e-1381-4740-a308-f86119f3bcd3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:39.578544 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 145/292] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:00:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 15:00:39.581065 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b735d8c-b376-4750-9f11-4fae1ce97e13 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:39.648640 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:39.690490 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b735d8c-b376-4750-9f11-4fae1ce97e13 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:39.726305 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b551cb56-b11e-4064-b671-c2dcb14803e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:39.727445 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b551cb56-b11e-4064-b671-c2dcb14803e3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:39.727899 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b551cb56-b11e-4064-b671-c2dcb14803e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:39.728330 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b551cb56-b11e-4064-b671-c2dcb14803e3 None admin] PATH_INFO: `/v3/projects/65801518f2424895ab7af78ae7acb341` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:39.729737 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b551cb56-b11e-4064-b671-c2dcb14803e3 None admin] RBAC: Authorizing `identity:get_project(project_id=65801518f2424895ab7af78ae7acb341)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:39.731410 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b551cb56-b11e-4064-b671-c2dcb14803e3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:39.736544 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 148/293] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:00:39 2022] GET /identity/v3/projects/65801518f2424895ab7af78ae7acb341 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:39.740338 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99da77e-1381-4740-a308-f86119f3bcd3 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:39.870966 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:39.877027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99da77e-1381-4740-a308-f86119f3bcd3 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:39.928705 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94e334a-8ae3-4510-85aa-98815cafa6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:39.929384 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e334a-8ae3-4510-85aa-98815cafa6a8 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:39.929668 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e334a-8ae3-4510-85aa-98815cafa6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:39.929949 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e334a-8ae3-4510-85aa-98815cafa6a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:39.930787 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94e334a-8ae3-4510-85aa-98815cafa6a8 None admin] RBAC: Adding query filter params (role.id=8442c42e847a4b38b76f3a7fbe9ada85, scope.project.id=65801518f2424895ab7af78ae7acb341, user.id=dc9d9a4819c74e949d95bae75a1084ed) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:39.931129 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94e334a-8ae3-4510-85aa-98815cafa6a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:39.932590 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94e334a-8ae3-4510-85aa-98815cafa6a8 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:39.944017 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 146/294] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:00:39 2022] GET /identity/v3/role_assignments?user.id=dc9d9a4819c74e949d95bae75a1084ed&scope.project.id=65801518f2424895ab7af78ae7acb341&role.id=8442c42e847a4b38b76f3a7fbe9ada85 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:40.030420 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5256ef-ca28-4782-8ac0-fc72115488db None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:40.030960 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5256ef-ca28-4782-8ac0-fc72115488db None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:40.031045 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5256ef-ca28-4782-8ac0-fc72115488db None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:40.033219 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 149/295] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:40.038783 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe30f452-f9a8-485b-894c-79767854a80f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:40.039160 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe30f452-f9a8-485b-894c-79767854a80f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:40.039454 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe30f452-f9a8-485b-894c-79767854a80f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:40.064025 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-fe30f452-f9a8-485b-894c-79767854a80f None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:40.065235 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fe30f452-f9a8-485b-894c-79767854a80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:40.149804 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 147/296] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:00:40.194082 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5256ef-ca28-4782-8ac0-fc72115488db None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:40.308671 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5256ef-ca28-4782-8ac0-fc72115488db None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:40.339534 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:40.345234 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d35f66-5507-4bf0-af42-cd82f44116fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:40.347605 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d35f66-5507-4bf0-af42-cd82f44116fe None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:40.347890 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d35f66-5507-4bf0-af42-cd82f44116fe None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:40.348165 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d35f66-5507-4bf0-af42-cd82f44116fe None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:40.360842 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d35f66-5507-4bf0-af42-cd82f44116fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:40.370602 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-20d35f66-5507-4bf0-af42-cd82f44116fe None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 15:00:40.370602 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:40.370602 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:40.370602 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:40.370602 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:40.370602 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:40.370602 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:40.370602 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:40.370602 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:40.370602 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:40.370602 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:40.370602 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:40.370602 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:40.370602 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:40.370602 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:40.370602 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:40.371659 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:40.371659 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:40.371659 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:40.371659 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:40.371659 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:40.371659 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:40.371659 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:40.371659 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:40.371659 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:40.371659 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:40.371659 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:40.371659 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:00:40.371659 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:40.371659 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:40.371659 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:40.371659 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:40.372492 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:40.372492 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:40.372492 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:40.372492 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:40.372492 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:40.372492 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:40.372492 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:40.372492 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:40.372492 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:40.372492 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:40.372492 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 15:00:40.372492 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:00:40.372492 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 150/297] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:00:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:00:40.378329 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe30f452-f9a8-485b-894c-79767854a80f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:40.444843 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff77fa-538b-4a52-a2f5-39f47c9d57bc None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:40.445166 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff77fa-538b-4a52-a2f5-39f47c9d57bc None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:40.445451 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff77fa-538b-4a52-a2f5-39f47c9d57bc None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:40.446826 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 151/298] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 15:00:40.452212 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5aefa8-336b-4fa1-8f30-e28763a99852 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:40.452505 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5aefa8-336b-4fa1-8f30-e28763a99852 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:40.452790 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5aefa8-336b-4fa1-8f30-e28763a99852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:40.471488 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-cd5aefa8-336b-4fa1-8f30-e28763a99852 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:40.472700 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cd5aefa8-336b-4fa1-8f30-e28763a99852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:40.497056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:40.508732 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe30f452-f9a8-485b-894c-79767854a80f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:40.556997 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1faa4a4-8b9e-4d62-8468-5334635387b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:40.557831 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1faa4a4-8b9e-4d62-8468-5334635387b9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:40.558212 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1faa4a4-8b9e-4d62-8468-5334635387b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:40.558586 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1faa4a4-8b9e-4d62-8468-5334635387b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:40.561912 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1faa4a4-8b9e-4d62-8468-5334635387b9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:40.561912 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1faa4a4-8b9e-4d62-8468-5334635387b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:40.563344 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1faa4a4-8b9e-4d62-8468-5334635387b9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:40.571492 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 152/299] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 15:00:40.574833 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 148/300] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 15:00:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:40.579965 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5aefa8-336b-4fa1-8f30-e28763a99852 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:40.593952 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1faa4a4-8b9e-4d62-8468-5334635387b9 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:40.688578 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5aefa8-336b-4fa1-8f30-e28763a99852 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:40.697234 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1faa4a4-8b9e-4d62-8468-5334635387b9 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:40.722113 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:40.736476 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c773e218-d0dc-45e8-92ee-344819b9276b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:40.737361 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c773e218-d0dc-45e8-92ee-344819b9276b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:40.737476 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9248c0d0-0cff-4b8a-98d7-6364113c1406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:40.737640 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c773e218-d0dc-45e8-92ee-344819b9276b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:40.737918 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c773e218-d0dc-45e8-92ee-344819b9276b None admin] PATH_INFO: `/v3/users/ac3b33aa9c8b4888b81cf4eb021666b8` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:40.738705 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9248c0d0-0cff-4b8a-98d7-6364113c1406 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:40.739153 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9248c0d0-0cff-4b8a-98d7-6364113c1406 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:40.739567 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9248c0d0-0cff-4b8a-98d7-6364113c1406 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:40.741900 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9248c0d0-0cff-4b8a-98d7-6364113c1406 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:40.743576 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9248c0d0-0cff-4b8a-98d7-6364113c1406 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:40.745690 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 149/301] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:00:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:00:40.747039 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b3cd0b-82e8-4ba0-85b2-f682597c120a None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:40.747513 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b3cd0b-82e8-4ba0-85b2-f682597c120a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:40.747958 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b3cd0b-82e8-4ba0-85b2-f682597c120a None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:40.752011 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 150/302] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:40 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:40.753459 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b3cd0b-82e8-4ba0-85b2-f682597c120a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:40.753962 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c773e218-d0dc-45e8-92ee-344819b9276b None admin] RBAC: Authorizing `identity:get_user(user_id=ac3b33aa9c8b4888b81cf4eb021666b8)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:40.755474 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c773e218-d0dc-45e8-92ee-344819b9276b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:40.758314 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 153/303] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:00:40 2022] GET /identity/v3/users/ac3b33aa9c8b4888b81cf4eb021666b8 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:40.763015 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa922823-7d3c-4d8a-9b76-5d359035280e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:40.763015 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa922823-7d3c-4d8a-9b76-5d359035280e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:40.763015 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa922823-7d3c-4d8a-9b76-5d359035280e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:40.777645 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:40.780667 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-aa922823-7d3c-4d8a-9b76-5d359035280e None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:40.782514 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-aa922823-7d3c-4d8a-9b76-5d359035280e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:40.860355 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 154/304] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:00:40.861652 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa922823-7d3c-4d8a-9b76-5d359035280e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:40.876416 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b3cd0b-82e8-4ba0-85b2-f682597c120a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:40.915009 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c1db48-c057-40be-b725-e6f644b9c8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:40.915681 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1db48-c057-40be-b725-e6f644b9c8ba None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:40.915962 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1db48-c057-40be-b725-e6f644b9c8ba None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:40.916444 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c1db48-c057-40be-b725-e6f644b9c8ba None admin] PATH_INFO: `/v3/groups` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:40.916920 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c1db48-c057-40be-b725-e6f644b9c8ba None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:40.918366 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c1db48-c057-40be-b725-e6f644b9c8ba None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:40.933661 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 151/305] 10.222.0.50 () {60 vars in 1134 bytes} [Fri May 27 15:00:40 2022] POST /identity/v3/groups => generated 229 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:00:40.944004 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c1db48-c057-40be-b725-e6f644b9c8ba None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:40.980173 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa922823-7d3c-4d8a-9b76-5d359035280e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:40.993336 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:41.024626 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa9c54-3051-4bf1-bf62-3bf77380c2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:41.025808 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa9c54-3051-4bf1-bf62-3bf77380c2be None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:41.026310 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa9c54-3051-4bf1-bf62-3bf77380c2be None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:41.026754 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa9c54-3051-4bf1-bf62-3bf77380c2be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:41.027934 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaa9c54-3051-4bf1-bf62-3bf77380c2be None admin] RBAC: Adding query filter params (role.id=1b313eb2327b44b7a11d92914db65bb9, user.id=ac3b33aa9c8b4888b81cf4eb021666b8, scope.system=all) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:41.028421 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaa9c54-3051-4bf1-bf62-3bf77380c2be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:41.030311 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaa9c54-3051-4bf1-bf62-3bf77380c2be None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:41.043388 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 155/306] 10.222.0.50 () {56 vars in 1308 bytes} [Fri May 27 15:00:40 2022] GET /identity/v3/role_assignments?user.id=ac3b33aa9c8b4888b81cf4eb021666b8&scope.system=all&role.id=1b313eb2327b44b7a11d92914db65bb9 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:41.062553 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:41.088319 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c1db48-c057-40be-b725-e6f644b9c8ba None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:41.130282 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c157d75e-1a20-4a10-83ce-f9673e34b72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:41.131055 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157d75e-1a20-4a10-83ce-f9673e34b72b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:41.131344 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157d75e-1a20-4a10-83ce-f9673e34b72b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:41.131620 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157d75e-1a20-4a10-83ce-f9673e34b72b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:41.144209 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c157d75e-1a20-4a10-83ce-f9673e34b72b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:41.152333 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-c157d75e-1a20-4a10-83ce-f9673e34b72b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 15:00:41.152333 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:41.152333 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:41.152333 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:41.152333 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:41.152333 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:41.152333 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:41.152333 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:41.152333 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:41.152333 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:41.152333 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:41.152333 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:41.152333 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:41.152333 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:41.152333 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:41.152333 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:41.153167 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:41.153167 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:41.153167 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:41.153167 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:41.153167 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:41.153167 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:41.153167 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:41.153167 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:41.153167 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:41.153167 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:41.153167 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:41.153167 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:00:41.153167 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:41.153167 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:41.153167 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:41.153167 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:41.154222 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:41.154222 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:41.154222 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:41.154222 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:41.154222 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:41.154222 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:41.154222 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:41.154222 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:41.154222 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:41.154222 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:41.154222 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 15:00:41.154222 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:00:41.154222 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 152/307] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:00:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:00:41.159701 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa9c54-3051-4bf1-bf62-3bf77380c2be None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:41.275371 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa9c54-3051-4bf1-bf62-3bf77380c2be None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:41.311585 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5f4af-555c-47b9-996f-5137d8fcf0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:41.312259 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5f4af-555c-47b9-996f-5137d8fcf0f5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:41.312543 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5f4af-555c-47b9-996f-5137d8fcf0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:41.313341 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5f4af-555c-47b9-996f-5137d8fcf0f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:41.314091 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa5f4af-555c-47b9-996f-5137d8fcf0f5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:41.314396 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa5f4af-555c-47b9-996f-5137d8fcf0f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:41.315785 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa5f4af-555c-47b9-996f-5137d8fcf0f5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:41.324950 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 156/308] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 15:00:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:41.330891 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c157d75e-1a20-4a10-83ce-f9673e34b72b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:41.434554 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c157d75e-1a20-4a10-83ce-f9673e34b72b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:41.469635 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1704c5e-c1eb-4081-b089-f33b13bb62de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:41.470908 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1704c5e-c1eb-4081-b089-f33b13bb62de None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:41.471395 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1704c5e-c1eb-4081-b089-f33b13bb62de None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:41.471870 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1704c5e-c1eb-4081-b089-f33b13bb62de None admin] PATH_INFO: `/v3/users/6f52f8f1850247cfbbb016d62c60de86` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:41.473289 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1704c5e-c1eb-4081-b089-f33b13bb62de None admin] RBAC: Authorizing `identity:get_user(user_id=6f52f8f1850247cfbbb016d62c60de86)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:41.474988 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1704c5e-c1eb-4081-b089-f33b13bb62de None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:41.477134 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 153/309] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:00:41 2022] GET /identity/v3/users/6f52f8f1850247cfbbb016d62c60de86 => generated 293 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:41.482461 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5f4af-555c-47b9-996f-5137d8fcf0f5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:41.523583 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:41.595212 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5f4af-555c-47b9-996f-5137d8fcf0f5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:41.634752 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f4756-ebf9-4d82-a48e-8319ce32a2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:41.635675 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f4756-ebf9-4d82-a48e-8319ce32a2f6 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:41.635958 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f4756-ebf9-4d82-a48e-8319ce32a2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:41.636232 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f4756-ebf9-4d82-a48e-8319ce32a2f6 None admin] PATH_INFO: `/v3/system/users/6f52f8f1850247cfbbb016d62c60de86/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:41.637766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296f4756-ebf9-4d82-a48e-8319ce32a2f6 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6f52f8f1850247cfbbb016d62c60de86, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:41.639175 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296f4756-ebf9-4d82-a48e-8319ce32a2f6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:41.649660 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 157/310] 10.222.0.50 () {58 vars in 1334 bytes} [Fri May 27 15:00:41 2022] PUT /identity/v3/system/users/6f52f8f1850247cfbbb016d62c60de86/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:00:41.936869 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3dd09-44a3-4c19-bd70-2394fca92616 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:41.937382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3dd09-44a3-4c19-bd70-2394fca92616 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:41.937457 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c3dd09-44a3-4c19-bd70-2394fca92616 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:41.939222 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 154/311] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:41.944789 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0c57eb-0faf-41c6-a437-c366631eb1ba None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:41.945088 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0c57eb-0faf-41c6-a437-c366631eb1ba None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:41.945372 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0c57eb-0faf-41c6-a437-c366631eb1ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:41.963354 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-ed0c57eb-0faf-41c6-a437-c366631eb1ba None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:41.964536 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ed0c57eb-0faf-41c6-a437-c366631eb1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:42.034558 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 158/312] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:00:42.057910 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c3dd09-44a3-4c19-bd70-2394fca92616 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:42.069592 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:42.169615 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c3dd09-44a3-4c19-bd70-2394fca92616 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:42.206748 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d5f006-59c7-4e14-8278-3676fb60d4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:42.207558 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5f006-59c7-4e14-8278-3676fb60d4de None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:42.207844 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5f006-59c7-4e14-8278-3676fb60d4de None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:42.208124 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d5f006-59c7-4e14-8278-3676fb60d4de None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:42.220463 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d5f006-59c7-4e14-8278-3676fb60d4de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:42.228436 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-92d5f006-59c7-4e14-8278-3676fb60d4de None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 15:00:42.228436 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:42.228436 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:42.228436 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:42.228436 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:42.228436 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:42.228436 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:42.228436 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:42.228436 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:42.228436 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:42.228436 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:42.228436 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:42.228436 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:42.228436 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:42.228436 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:42.228436 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:42.229427 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:42.229427 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:42.229427 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:42.229427 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:42.229427 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:42.229427 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:42.229427 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:42.229427 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:42.229427 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:42.229427 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:42.229427 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:42.229427 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:00:42.229427 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:42.229427 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:42.229427 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:42.229427 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:42.230239 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:42.230239 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:42.230239 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:42.230239 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:42.230239 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:42.230239 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:42.230239 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:42.230239 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:42.230239 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:42.230239 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:42.230239 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 15:00:42.230239 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:00:42.230239 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 155/313] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:00:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:00:42.237873 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0c57eb-0faf-41c6-a437-c366631eb1ba None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:42.271321 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:42.347313 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0c57eb-0faf-41c6-a437-c366631eb1ba None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:42.382698 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20db4ac9-db3a-4899-82cb-64fe1c3acdff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:42.383761 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db4ac9-db3a-4899-82cb-64fe1c3acdff None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:42.384250 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db4ac9-db3a-4899-82cb-64fe1c3acdff None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:42.384723 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db4ac9-db3a-4899-82cb-64fe1c3acdff None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:42.385819 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20db4ac9-db3a-4899-82cb-64fe1c3acdff None admin] RBAC: Adding query filter params (name=reader) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:42.386354 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20db4ac9-db3a-4899-82cb-64fe1c3acdff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:42.387977 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20db4ac9-db3a-4899-82cb-64fe1c3acdff None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:42.402525 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 159/314] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 15:00:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:42.408450 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d5f006-59c7-4e14-8278-3676fb60d4de None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:42.476958 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:42.512172 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d5f006-59c7-4e14-8278-3676fb60d4de None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:42.550626 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6bd151-8a58-48f6-86be-b24aea4c534f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:42.551531 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6bd151-8a58-48f6-86be-b24aea4c534f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:42.551809 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6bd151-8a58-48f6-86be-b24aea4c534f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:42.552083 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6bd151-8a58-48f6-86be-b24aea4c534f None admin] PATH_INFO: `/v3/users/8ee4b55c12aa4a35904984a26a6da877` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:42.553253 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6bd151-8a58-48f6-86be-b24aea4c534f None admin] RBAC: Authorizing `identity:get_user(user_id=8ee4b55c12aa4a35904984a26a6da877)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:42.554681 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6bd151-8a58-48f6-86be-b24aea4c534f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:42.556612 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 156/315] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:00:42 2022] GET /identity/v3/users/8ee4b55c12aa4a35904984a26a6da877 => generated 297 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:00:42.564622 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20db4ac9-db3a-4899-82cb-64fe1c3acdff None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:42.666324 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:42.675980 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20db4ac9-db3a-4899-82cb-64fe1c3acdff None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:42.711478 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221ba59c-b9ea-4a30-901f-b5ebcdc74ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:42.712131 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ba59c-b9ea-4a30-901f-b5ebcdc74ce7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:42.712415 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ba59c-b9ea-4a30-901f-b5ebcdc74ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:42.712944 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ba59c-b9ea-4a30-901f-b5ebcdc74ce7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:42.783644 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221ba59c-b9ea-4a30-901f-b5ebcdc74ce7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:42.785138 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221ba59c-b9ea-4a30-901f-b5ebcdc74ce7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:42.889528 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:42.900056 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 160/316] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:00:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 15:00:42.905772 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6bd151-8a58-48f6-86be-b24aea4c534f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:43.008832 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6bd151-8a58-48f6-86be-b24aea4c534f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:43.043037 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e813836f-c27f-41f6-9571-b11b3740cbc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:43.043998 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813836f-c27f-41f6-9571-b11b3740cbc5 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:43.044280 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813836f-c27f-41f6-9571-b11b3740cbc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:43.044550 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e813836f-c27f-41f6-9571-b11b3740cbc5 None admin] PATH_INFO: `/v3/projects/65801518f2424895ab7af78ae7acb341` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:43.045739 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e813836f-c27f-41f6-9571-b11b3740cbc5 None admin] RBAC: Authorizing `identity:get_project(project_id=65801518f2424895ab7af78ae7acb341)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:43.047750 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e813836f-c27f-41f6-9571-b11b3740cbc5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:43.049615 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 157/317] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:00:42 2022] GET /identity/v3/projects/65801518f2424895ab7af78ae7acb341 => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:43.055238 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221ba59c-b9ea-4a30-901f-b5ebcdc74ce7 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:43.176186 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221ba59c-b9ea-4a30-901f-b5ebcdc74ce7 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:43.211070 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9298ec16-7db0-472f-83f4-13d4669b4d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:43.211735 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298ec16-7db0-472f-83f4-13d4669b4d8a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:43.212015 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298ec16-7db0-472f-83f4-13d4669b4d8a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:43.212284 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298ec16-7db0-472f-83f4-13d4669b4d8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:43.213368 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9298ec16-7db0-472f-83f4-13d4669b4d8a None admin] RBAC: Adding query filter params (role.id=1b313eb2327b44b7a11d92914db65bb9, scope.project.id=65801518f2424895ab7af78ae7acb341, user.id=8ee4b55c12aa4a35904984a26a6da877) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:43.213674 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9298ec16-7db0-472f-83f4-13d4669b4d8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:43.215135 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9298ec16-7db0-472f-83f4-13d4669b4d8a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:43.221885 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 161/318] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:00:43 2022] GET /identity/v3/role_assignments?user.id=8ee4b55c12aa4a35904984a26a6da877&scope.project.id=65801518f2424895ab7af78ae7acb341&role.id=1b313eb2327b44b7a11d92914db65bb9 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:43.323983 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03bbb04-afdf-4b7e-998f-038b1d04eac3 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:43.324289 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03bbb04-afdf-4b7e-998f-038b1d04eac3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:43.324479 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03bbb04-afdf-4b7e-998f-038b1d04eac3 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:43.326247 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 158/319] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:43.331896 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81741a9-de23-44a9-8b46-fb1de0341eff None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:43.332397 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81741a9-de23-44a9-8b46-fb1de0341eff None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:43.332473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81741a9-de23-44a9-8b46-fb1de0341eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:43.351409 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a81741a9-de23-44a9-8b46-fb1de0341eff None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:43.352608 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a81741a9-de23-44a9-8b46-fb1de0341eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:43.412529 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:43.428266 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 162/320] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:00:43.448204 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7218948-8126-4340-ba83-9e455c8bb38f None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:43.448507 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7218948-8126-4340-ba83-9e455c8bb38f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:43.448757 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7218948-8126-4340-ba83-9e455c8bb38f None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:43.450255 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 159/321] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:43.450973 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81741a9-de23-44a9-8b46-fb1de0341eff None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:43.455850 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84bd364-3445-43f4-8b13-5439b4b547b9 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:43.456131 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84bd364-3445-43f4-8b13-5439b4b547b9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:43.456403 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84bd364-3445-43f4-8b13-5439b4b547b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:43.464083 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:43.476213 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c84bd364-3445-43f4-8b13-5439b4b547b9 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:43.477384 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c84bd364-3445-43f4-8b13-5439b4b547b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:43.570413 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81741a9-de23-44a9-8b46-fb1de0341eff None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:43.571377 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 160/322] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 15:00:43.593849 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84bd364-3445-43f4-8b13-5439b4b547b9 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:43.607170 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969ca802-5638-423b-ab95-62f61afd5e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:43.607968 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ca802-5638-423b-ab95-62f61afd5e71 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:43.608247 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ca802-5638-423b-ab95-62f61afd5e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:43.608517 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ca802-5638-423b-ab95-62f61afd5e71 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:43.609751 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969ca802-5638-423b-ab95-62f61afd5e71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:43.611223 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969ca802-5638-423b-ab95-62f61afd5e71 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:43.612829 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 163/323] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:00:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:00:43.618661 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969ca802-5638-423b-ab95-62f61afd5e71 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:43.679732 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:43.702139 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84bd364-3445-43f4-8b13-5439b4b547b9 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:43.729590 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969ca802-5638-423b-ab95-62f61afd5e71 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:43.738712 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde7fa9-ee12-4b7a-8d06-35926fc73e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:43.739571 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde7fa9-ee12-4b7a-8d06-35926fc73e6e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:43.740235 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde7fa9-ee12-4b7a-8d06-35926fc73e6e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:43.740235 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde7fa9-ee12-4b7a-8d06-35926fc73e6e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:43.742315 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:43.754181 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cde7fa9-ee12-4b7a-8d06-35926fc73e6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:43.762141 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-4cde7fa9-ee12-4b7a-8d06-35926fc73e6e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 15:00:43.762141 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:43.762141 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:43.762141 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:43.762141 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:43.762141 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:43.762141 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:43.762141 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:43.762141 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:43.762141 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:43.762141 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:43.762141 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:43.762141 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:43.762141 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:43.762141 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:43.762141 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:43.763305 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:43.763305 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:43.763305 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:43.763305 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:43.763305 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:43.763305 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:43.763305 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:43.763305 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:43.763305 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:43.763305 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:43.763305 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:43.763305 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:00:43.763305 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:43.763305 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:43.763305 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:43.763305 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:43.764097 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:43.764097 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:43.764097 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:43.764097 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:43.764097 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:43.764097 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:43.764097 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:43.764097 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:43.764097 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:43.764097 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:43.764097 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 15:00:43.764097 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:00:43.764097 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 161/324] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:00:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:00:43.771359 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde7fa9-ee12-4b7a-8d06-35926fc73e6e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:43.773162 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7670903-6766-4cea-8178-e83ef68f1fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:43.773856 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7670903-6766-4cea-8178-e83ef68f1fe3 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:43.774421 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7670903-6766-4cea-8178-e83ef68f1fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:43.774701 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7670903-6766-4cea-8178-e83ef68f1fe3 None admin] PATH_INFO: `/v3/groups` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:43.776381 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7670903-6766-4cea-8178-e83ef68f1fe3 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:43.777866 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7670903-6766-4cea-8178-e83ef68f1fe3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:43.795961 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 164/325] 10.222.0.50 () {60 vars in 1134 bytes} [Fri May 27 15:00:43 2022] POST /identity/v3/groups => generated 226 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:00:43.899351 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde7fa9-ee12-4b7a-8d06-35926fc73e6e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:43.936104 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556784c4-1b00-45a1-912e-bccb37f9516b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:43.936846 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556784c4-1b00-45a1-912e-bccb37f9516b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:43.937133 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556784c4-1b00-45a1-912e-bccb37f9516b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:43.937408 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556784c4-1b00-45a1-912e-bccb37f9516b None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:43.938744 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556784c4-1b00-45a1-912e-bccb37f9516b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:43.939090 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556784c4-1b00-45a1-912e-bccb37f9516b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:43.940513 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556784c4-1b00-45a1-912e-bccb37f9516b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:43.949064 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e55104-eae6-4060-be6a-2e7f7a717449 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:43.949371 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e55104-eae6-4060-be6a-2e7f7a717449 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:43.949641 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e55104-eae6-4060-be6a-2e7f7a717449 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:43.951271 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 165/326] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 15:00:43.955944 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 162/327] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 15:00:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:00:43.956109 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e55104-eae6-4060-be6a-2e7f7a717449 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:43.956819 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5255c8-27fc-451a-9fff-3496cdc02888 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:43.957113 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5255c8-27fc-451a-9fff-3496cdc02888 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:43.957393 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5255c8-27fc-451a-9fff-3496cdc02888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:43.980368 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5d5255c8-27fc-451a-9fff-3496cdc02888 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:43.983152 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5d5255c8-27fc-451a-9fff-3496cdc02888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:44.048096 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:44.064387 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 163/328] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 15:00:44.083241 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e55104-eae6-4060-be6a-2e7f7a717449 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:44.088246 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5255c8-27fc-451a-9fff-3496cdc02888 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:44.099656 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:44.129392 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38d39c5-ae99-4890-8fc1-ae35d3e4af7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:44.130275 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d39c5-ae99-4890-8fc1-ae35d3e4af7b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:44.130557 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d39c5-ae99-4890-8fc1-ae35d3e4af7b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:44.130831 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d39c5-ae99-4890-8fc1-ae35d3e4af7b None admin] PATH_INFO: `/v3/users/ac3b33aa9c8b4888b81cf4eb021666b8` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:44.132058 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38d39c5-ae99-4890-8fc1-ae35d3e4af7b None admin] RBAC: Authorizing `identity:get_user(user_id=ac3b33aa9c8b4888b81cf4eb021666b8)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:44.133514 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38d39c5-ae99-4890-8fc1-ae35d3e4af7b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:44.135380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 166/329] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:00:43 2022] GET /identity/v3/users/ac3b33aa9c8b4888b81cf4eb021666b8 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:44.140908 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38d39c5-ae99-4890-8fc1-ae35d3e4af7b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:44.227228 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5255c8-27fc-451a-9fff-3496cdc02888 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:44.274521 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a76b42-9244-467e-938c-b01d1afb6b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:44.274521 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a76b42-9244-467e-938c-b01d1afb6b12 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:44.274802 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a76b42-9244-467e-938c-b01d1afb6b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:44.274939 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a76b42-9244-467e-938c-b01d1afb6b12 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:44.279415 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38d39c5-ae99-4890-8fc1-ae35d3e4af7b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:44.287331 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a76b42-9244-467e-938c-b01d1afb6b12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:44.298931 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-23a76b42-9244-467e-938c-b01d1afb6b12 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 15:00:44.298931 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:44.298931 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:44.298931 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:44.298931 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:44.298931 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:44.298931 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:44.298931 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:44.298931 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:44.298931 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:44.298931 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:44.298931 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:44.298931 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:44.298931 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:44.298931 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:44.298931 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:44.300036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:44.300036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:44.300036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:44.300036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:44.300036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:44.300036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:44.300036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:44.300036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:44.300036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:44.300036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:44.300036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:44.300036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:00:44.300036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:44.300036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:44.300036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:44.300036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:44.301042 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:44.301042 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:44.301042 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:44.301042 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:44.301042 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:44.301042 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:44.301042 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:44.301042 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:44.301042 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:44.301042 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:44.301042 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 15:00:44.301042 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:00:44.301042 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 164/330] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:00:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:00:44.305231 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a76b42-9244-467e-938c-b01d1afb6b12 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:44.329351 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcd3ec6-0fb0-4c09-98ec-b7997737cae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:44.330515 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd3ec6-0fb0-4c09-98ec-b7997737cae5 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:44.330989 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd3ec6-0fb0-4c09-98ec-b7997737cae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:44.331523 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd3ec6-0fb0-4c09-98ec-b7997737cae5 None admin] PATH_INFO: `/v3/system/users/ac3b33aa9c8b4888b81cf4eb021666b8/roles/1b313eb2327b44b7a11d92914db65bb9` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:44.333206 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcd3ec6-0fb0-4c09-98ec-b7997737cae5 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ac3b33aa9c8b4888b81cf4eb021666b8, role_id=1b313eb2327b44b7a11d92914db65bb9)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:44.335875 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcd3ec6-0fb0-4c09-98ec-b7997737cae5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:44.350921 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 167/331] 10.222.0.50 () {58 vars in 1334 bytes} [Fri May 27 15:00:44 2022] PUT /identity/v3/system/users/ac3b33aa9c8b4888b81cf4eb021666b8/roles/1b313eb2327b44b7a11d92914db65bb9 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:00:44.370253 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:44.435857 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a76b42-9244-467e-938c-b01d1afb6b12 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:44.477797 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1952e9c2-602e-4307-a903-7df6840fa02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:44.478484 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1952e9c2-602e-4307-a903-7df6840fa02f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:44.478765 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1952e9c2-602e-4307-a903-7df6840fa02f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:44.479070 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1952e9c2-602e-4307-a903-7df6840fa02f None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:44.479876 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1952e9c2-602e-4307-a903-7df6840fa02f None admin] RBAC: Adding query filter params (name=member) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:44.480195 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1952e9c2-602e-4307-a903-7df6840fa02f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:44.481762 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1952e9c2-602e-4307-a903-7df6840fa02f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:44.492541 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 165/332] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 15:00:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:44.498181 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcd3ec6-0fb0-4c09-98ec-b7997737cae5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:44.553144 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:44.618594 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcd3ec6-0fb0-4c09-98ec-b7997737cae5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:44.661957 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60707ace-c102-4e6b-b35b-316ecc65dab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:44.662842 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60707ace-c102-4e6b-b35b-316ecc65dab1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:44.663192 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60707ace-c102-4e6b-b35b-316ecc65dab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:44.663473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60707ace-c102-4e6b-b35b-316ecc65dab1 None admin] PATH_INFO: `/v3/users/6f52f8f1850247cfbbb016d62c60de86` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:44.664857 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60707ace-c102-4e6b-b35b-316ecc65dab1 None admin] RBAC: Authorizing `identity:get_user(user_id=6f52f8f1850247cfbbb016d62c60de86)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:44.666310 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60707ace-c102-4e6b-b35b-316ecc65dab1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:44.668171 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 168/333] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:00:44 2022] GET /identity/v3/users/6f52f8f1850247cfbbb016d62c60de86 => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:44.674197 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1952e9c2-602e-4307-a903-7df6840fa02f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:44.808159 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1952e9c2-602e-4307-a903-7df6840fa02f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:44.818291 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:44.857400 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b306848-ca49-49ee-a4e9-da2fbd768e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:44.858077 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b306848-ca49-49ee-a4e9-da2fbd768e52 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:44.858364 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b306848-ca49-49ee-a4e9-da2fbd768e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:44.858643 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b306848-ca49-49ee-a4e9-da2fbd768e52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:44.859509 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b306848-ca49-49ee-a4e9-da2fbd768e52 None admin] RBAC: Adding query filter params (role.id=8442c42e847a4b38b76f3a7fbe9ada85, user.id=6f52f8f1850247cfbbb016d62c60de86, scope.system=all) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:44.860420 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b306848-ca49-49ee-a4e9-da2fbd768e52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:44.861887 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b306848-ca49-49ee-a4e9-da2fbd768e52 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:44.875010 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 166/334] 10.222.0.50 () {56 vars in 1308 bytes} [Fri May 27 15:00:44 2022] GET /identity/v3/role_assignments?user.id=6f52f8f1850247cfbbb016d62c60de86&scope.system=all&role.id=8442c42e847a4b38b76f3a7fbe9ada85 => generated 510 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:00:46.343657 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa23036-8bf4-465a-a1a2-7a07a1bbd2e3 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:46.344210 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa23036-8bf4-465a-a1a2-7a07a1bbd2e3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:46.344363 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa23036-8bf4-465a-a1a2-7a07a1bbd2e3 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:46.345726 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 169/335] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:46.351140 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec70b4-1976-468a-8bbc-7dd94af708b6 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:46.351558 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec70b4-1976-468a-8bbc-7dd94af708b6 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:46.351887 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec70b4-1976-468a-8bbc-7dd94af708b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:46.376705 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c2ec70b4-1976-468a-8bbc-7dd94af708b6 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:46.377880 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c2ec70b4-1976-468a-8bbc-7dd94af708b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:46.460418 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 167/336] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:00:46.494869 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa23036-8bf4-465a-a1a2-7a07a1bbd2e3 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:46.621678 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa23036-8bf4-465a-a1a2-7a07a1bbd2e3 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:46.639724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:46.662253 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cf1ed8-2201-4079-9963-d173e7bc87e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:46.663067 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf1ed8-2201-4079-9963-d173e7bc87e4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:46.663348 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf1ed8-2201-4079-9963-d173e7bc87e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:46.663621 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cf1ed8-2201-4079-9963-d173e7bc87e4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:46.677680 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cf1ed8-2201-4079-9963-d173e7bc87e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:46.685361 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-03cf1ed8-2201-4079-9963-d173e7bc87e4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 15:00:46.685361 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:46.685361 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:46.685361 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:46.685361 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:46.685361 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:46.685361 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:46.685361 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:46.685361 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:46.685361 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:46.685361 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:46.685361 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:46.685361 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:46.685361 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:46.685361 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:46.685361 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:46.686232 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:46.686232 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:46.686232 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:46.686232 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:46.686232 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:46.686232 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:46.686232 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:46.686232 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:46.686232 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:46.686232 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:46.686232 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:46.686232 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:00:46.686232 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:46.686232 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:46.686232 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:46.686232 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:46.687049 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:46.687049 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:46.687049 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:46.687049 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:46.687049 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:46.687049 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:46.687049 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:46.687049 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:46.687049 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:46.687049 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:46.687049 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 15:00:46.687049 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:00:46.687049 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 170/337] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:00:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:00:46.693310 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec70b4-1976-468a-8bbc-7dd94af708b6 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:46.803266 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:46.811875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec70b4-1976-468a-8bbc-7dd94af708b6 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:46.860747 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545e217b-646a-4cdb-8b41-0ac5a32d9556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:46.861448 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e217b-646a-4cdb-8b41-0ac5a32d9556 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:46.861735 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e217b-646a-4cdb-8b41-0ac5a32d9556 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:46.861967 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e217b-646a-4cdb-8b41-0ac5a32d9556 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:46.862731 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545e217b-646a-4cdb-8b41-0ac5a32d9556 None admin] RBAC: Adding query filter params (name=member) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:46.863056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545e217b-646a-4cdb-8b41-0ac5a32d9556 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:46.864434 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545e217b-646a-4cdb-8b41-0ac5a32d9556 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:46.880395 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 168/338] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 15:00:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:46.885909 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cf1ed8-2201-4079-9963-d173e7bc87e4 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:47.025184 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cf1ed8-2201-4079-9963-d173e7bc87e4 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:47.041139 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e31c6-b947-4328-b934-0787be6e4e8d None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:47.041442 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e31c6-b947-4328-b934-0787be6e4e8d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:47.041701 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058e31c6-b947-4328-b934-0787be6e4e8d None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:47.043757 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 169/339] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:47.049196 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c5a4c4-8947-4582-bdb3-4710ed9bae2c None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:47.049482 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c5a4c4-8947-4582-bdb3-4710ed9bae2c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:47.049758 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c5a4c4-8947-4582-bdb3-4710ed9bae2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:47.084537 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d6c5a4c4-8947-4582-bdb3-4710ed9bae2c None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:47.085975 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d6c5a4c4-8947-4582-bdb3-4710ed9bae2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:47.106550 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b967e2e-a341-43da-86f0-b3e1dfc00ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:47.107610 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b967e2e-a341-43da-86f0-b3e1dfc00ccf None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:47.108094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b967e2e-a341-43da-86f0-b3e1dfc00ccf None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:47.108587 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b967e2e-a341-43da-86f0-b3e1dfc00ccf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:47.196640 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:47.222206 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 170/340] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:00:47.247788 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b967e2e-a341-43da-86f0-b3e1dfc00ccf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:47.249253 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b967e2e-a341-43da-86f0-b3e1dfc00ccf None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:47.261082 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c5a4c4-8947-4582-bdb3-4710ed9bae2c None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:47.466950 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:47.489150 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 171/341] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:00:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 15:00:47.494836 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b967e2e-a341-43da-86f0-b3e1dfc00ccf None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:47.496876 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c5a4c4-8947-4582-bdb3-4710ed9bae2c None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:47.578984 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5073dd60-cf4d-4d71-9f00-3eef1cfdfa4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:47.580342 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073dd60-cf4d-4d71-9f00-3eef1cfdfa4c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:47.580812 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073dd60-cf4d-4d71-9f00-3eef1cfdfa4c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:47.581307 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5073dd60-cf4d-4d71-9f00-3eef1cfdfa4c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:47.602997 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5073dd60-cf4d-4d71-9f00-3eef1cfdfa4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:47.616224 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-5073dd60-cf4d-4d71-9f00-3eef1cfdfa4c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 15:00:47.616224 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:47.616224 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:47.616224 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:47.616224 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:47.616224 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:47.616224 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:47.616224 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:47.616224 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:47.616224 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:47.616224 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:47.616224 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:47.616224 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:47.616224 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:47.616224 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:47.616224 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:47.617174 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:47.617174 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:47.617174 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:47.617174 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:47.617174 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:47.617174 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:47.617174 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:47.617174 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:47.617174 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:47.617174 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:47.617174 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:47.617174 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:00:47.617174 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:47.617174 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:47.617174 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:47.617174 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:47.617998 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:47.617998 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:47.617998 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:47.617998 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:47.617998 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:47.617998 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:47.617998 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:47.617998 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:47.617998 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:47.617998 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:47.617998 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 15:00:47.617998 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:00:47.621406 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 171/342] 10.222.0.50 () {56 vars in 1114 bytes} [Fri May 27 15:00:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:00:47.626700 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e675a14b-4c6f-479c-beda-c817b674d695 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:47.628454 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e675a14b-4c6f-479c-beda-c817b674d695 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:47.629560 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e675a14b-4c6f-479c-beda-c817b674d695 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:47.631721 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 172/343] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:47 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:47.633788 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e675a14b-4c6f-479c-beda-c817b674d695 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:47.657559 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:47.716764 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b967e2e-a341-43da-86f0-b3e1dfc00ccf None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:47.784461 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e675a14b-4c6f-479c-beda-c817b674d695 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:47.826332 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b2253-5cbb-4e34-935a-169789a7bac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:47.827368 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b2253-5cbb-4e34-935a-169789a7bac1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:47.827494 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c66de9-8adc-4230-aa17-ff162eb7e505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:47.827920 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b2253-5cbb-4e34-935a-169789a7bac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:47.828839 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c66de9-8adc-4230-aa17-ff162eb7e505 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:47.829077 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2b2253-5cbb-4e34-935a-169789a7bac1 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:47.829256 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c66de9-8adc-4230-aa17-ff162eb7e505 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:47.829417 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c66de9-8adc-4230-aa17-ff162eb7e505 None admin] PATH_INFO: `/v3/projects/00203c7cc02341ee8ee7a591095f0c7d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:47.830495 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2b2253-5cbb-4e34-935a-169789a7bac1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:47.830619 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c66de9-8adc-4230-aa17-ff162eb7e505 None admin] RBAC: Authorizing `identity:get_project(project_id=00203c7cc02341ee8ee7a591095f0c7d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:47.831082 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2b2253-5cbb-4e34-935a-169789a7bac1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:47.832074 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c66de9-8adc-4230-aa17-ff162eb7e505 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:47.832779 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2b2253-5cbb-4e34-935a-169789a7bac1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:47.833847 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 172/344] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:00:47 2022] GET /identity/v3/projects/00203c7cc02341ee8ee7a591095f0c7d => generated 298 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:47.840278 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038236d-d5cb-4e72-b309-afd314146846 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:47.840572 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038236d-d5cb-4e72-b309-afd314146846 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:47.840912 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038236d-d5cb-4e72-b309-afd314146846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:47.844591 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 173/345] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:00:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:47.845705 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b2253-5cbb-4e34-935a-169789a7bac1 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:47.871883 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f038236d-d5cb-4e72-b309-afd314146846 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:47.872416 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:47.873064 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f038236d-d5cb-4e72-b309-afd314146846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:47.971245 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2b2253-5cbb-4e34-935a-169789a7bac1 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:48.009204 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ae6bbd-5605-4473-9d59-4e0ad099e26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:48.010482 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ae6bbd-5605-4473-9d59-4e0ad099e26c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:48.010948 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ae6bbd-5605-4473-9d59-4e0ad099e26c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:48.011470 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ae6bbd-5605-4473-9d59-4e0ad099e26c None admin] PATH_INFO: `/v3/groups/d49ca26b382043a0a226f9249c3326c8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:48.019451 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ae6bbd-5605-4473-9d59-4e0ad099e26c None admin] RBAC: Authorizing `identity:get_group(group_id=d49ca26b382043a0a226f9249c3326c8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:48.020529 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 173/346] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:00:48.021556 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f038236d-d5cb-4e72-b309-afd314146846 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:48.022537 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ae6bbd-5605-4473-9d59-4e0ad099e26c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:48.024558 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 174/347] 10.222.0.50 () {56 vars in 1180 bytes} [Fri May 27 15:00:47 2022] GET /identity/v3/groups/d49ca26b382043a0a226f9249c3326c8 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:48.031113 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ae6bbd-5605-4473-9d59-4e0ad099e26c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:48.127360 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:48.152570 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f038236d-d5cb-4e72-b309-afd314146846 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:48.190783 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3628d554-04c8-4a39-aad2-009f8ed51b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:48.191742 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628d554-04c8-4a39-aad2-009f8ed51b86 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:48.192185 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628d554-04c8-4a39-aad2-009f8ed51b86 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:48.192618 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628d554-04c8-4a39-aad2-009f8ed51b86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:48.195300 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:48.228925 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ae6bbd-5605-4473-9d59-4e0ad099e26c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:48.266952 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ddc409-7142-41d1-a168-7fc3354a2bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:48.268032 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ddc409-7142-41d1-a168-7fc3354a2bf3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:48.269122 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ddc409-7142-41d1-a168-7fc3354a2bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:48.269603 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ddc409-7142-41d1-a168-7fc3354a2bf3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:48.270725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ddc409-7142-41d1-a168-7fc3354a2bf3 None admin] RBAC: Adding query filter params (group.id=d49ca26b382043a0a226f9249c3326c8, role.id=8442c42e847a4b38b76f3a7fbe9ada85, scope.project.id=00203c7cc02341ee8ee7a591095f0c7d) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:48.271293 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ddc409-7142-41d1-a168-7fc3354a2bf3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:48.276182 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ddc409-7142-41d1-a168-7fc3354a2bf3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:48.278679 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3628d554-04c8-4a39-aad2-009f8ed51b86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:48.282012 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3628d554-04c8-4a39-aad2-009f8ed51b86 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:48.290238 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 175/348] 10.222.0.50 () {56 vars in 1376 bytes} [Fri May 27 15:00:48 2022] GET /identity/v3/role_assignments?group.id=d49ca26b382043a0a226f9249c3326c8&scope.project.id=00203c7cc02341ee8ee7a591095f0c7d&role.id=8442c42e847a4b38b76f3a7fbe9ada85 => generated 260 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:48.291462 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ddc409-7142-41d1-a168-7fc3354a2bf3 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:48.375428 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:48.423485 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 174/349] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:00:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 15:00:48.425286 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0138000d-99eb-4bf4-b056-a748498f3016 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:48.426086 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0138000d-99eb-4bf4-b056-a748498f3016 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:48.426577 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0138000d-99eb-4bf4-b056-a748498f3016 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:48.429081 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 175/350] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:48.430341 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0138000d-99eb-4bf4-b056-a748498f3016 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:48.511798 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ddc409-7142-41d1-a168-7fc3354a2bf3 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:48.552529 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57786c24-826f-4606-80b4-a578df1419b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:48.553747 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57786c24-826f-4606-80b4-a578df1419b5 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:48.554240 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57786c24-826f-4606-80b4-a578df1419b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:48.554720 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57786c24-826f-4606-80b4-a578df1419b5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:48.564173 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:48.570455 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57786c24-826f-4606-80b4-a578df1419b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:48.580613 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-57786c24-826f-4606-80b4-a578df1419b5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 27 15:00:48.580613 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:48.580613 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:48.580613 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:48.580613 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:48.580613 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:48.580613 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:48.580613 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:48.580613 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:48.580613 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:48.580613 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:48.580613 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:48.580613 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:48.580613 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:48.580613 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:48.580613 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:48.581570 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:48.581570 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:48.581570 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:48.581570 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:48.581570 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:48.581570 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:48.581570 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:48.581570 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:48.581570 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:48.581570 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:48.581570 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:48.581570 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:00:48.581570 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:48.581570 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:48.581570 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:48.581570 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:48.582369 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:48.582369 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:48.582369 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:48.582369 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:48.582369 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:48.582369 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:48.582369 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:48.582369 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:48.582369 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:48.582369 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:48.582369 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 27 15:00:48.582369 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:00:48.590102 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 176/351] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:00:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:00:48.592115 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29e9b6-df7a-4be2-98b5-1d2fc5cea72c None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:48.592607 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29e9b6-df7a-4be2-98b5-1d2fc5cea72c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:48.593089 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29e9b6-df7a-4be2-98b5-1d2fc5cea72c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:48.613119 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4b29e9b6-df7a-4be2-98b5-1d2fc5cea72c None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:48.614632 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4b29e9b6-df7a-4be2-98b5-1d2fc5cea72c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:48.639380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0138000d-99eb-4bf4-b056-a748498f3016 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:48.697843 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dd4dbe-9772-4338-89b4-568451063314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:48.699676 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd4dbe-9772-4338-89b4-568451063314 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:48.700368 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd4dbe-9772-4338-89b4-568451063314 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:48.701035 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dd4dbe-9772-4338-89b4-568451063314 None admin] PATH_INFO: `/v3/projects/00203c7cc02341ee8ee7a591095f0c7d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:48.701418 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 177/352] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:00:48.702491 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b29e9b6-df7a-4be2-98b5-1d2fc5cea72c None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:48.703870 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dd4dbe-9772-4338-89b4-568451063314 None admin] RBAC: Authorizing `identity:get_project(project_id=00203c7cc02341ee8ee7a591095f0c7d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:48.705718 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dd4dbe-9772-4338-89b4-568451063314 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:48.707759 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 176/353] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:00:48 2022] GET /identity/v3/projects/00203c7cc02341ee8ee7a591095f0c7d => generated 298 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:48.709162 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665cfef-c3a9-472c-8ef0-b85ab1be0e33 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:48.709595 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665cfef-c3a9-472c-8ef0-b85ab1be0e33 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:48.710022 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665cfef-c3a9-472c-8ef0-b85ab1be0e33 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:48.712157 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 177/354] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:48.720894 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40647d67-1834-4e24-a650-be28e04e2a8f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:48.721516 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40647d67-1834-4e24-a650-be28e04e2a8f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:48.722019 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40647d67-1834-4e24-a650-be28e04e2a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:48.744601 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-40647d67-1834-4e24-a650-be28e04e2a8f None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:48.746359 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-40647d67-1834-4e24-a650-be28e04e2a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:48.787247 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:48.818234 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b29e9b6-df7a-4be2-98b5-1d2fc5cea72c None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:48.826694 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 178/355] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:00:48.827901 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40647d67-1834-4e24-a650-be28e04e2a8f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:48.847826 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:48.857134 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401884c2-0d54-43d5-afcd-371df69d1bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:48.857816 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401884c2-0d54-43d5-afcd-371df69d1bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:48.858101 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401884c2-0d54-43d5-afcd-371df69d1bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:48.858382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401884c2-0d54-43d5-afcd-371df69d1bc2 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:48.859297 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401884c2-0d54-43d5-afcd-371df69d1bc2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:48.860252 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401884c2-0d54-43d5-afcd-371df69d1bc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:48.862076 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401884c2-0d54-43d5-afcd-371df69d1bc2 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:48.871410 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 178/356] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 15:00:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:48.872985 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401884c2-0d54-43d5-afcd-371df69d1bc2 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:48.956281 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40647d67-1834-4e24-a650-be28e04e2a8f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:48.985466 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401884c2-0d54-43d5-afcd-371df69d1bc2 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:48.996071 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2041f9-4b07-42f2-9926-033539bda88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:48.997372 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2041f9-4b07-42f2-9926-033539bda88f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:48.997862 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2041f9-4b07-42f2-9926-033539bda88f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:48.998340 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2041f9-4b07-42f2-9926-033539bda88f None admin] PATH_INFO: `/v3/groups/d49ca26b382043a0a226f9249c3326c8` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:48.999857 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2041f9-4b07-42f2-9926-033539bda88f None admin] RBAC: Authorizing `identity:get_group(group_id=d49ca26b382043a0a226f9249c3326c8)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:49.001840 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2041f9-4b07-42f2-9926-033539bda88f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:49.003951 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 179/357] 10.222.0.50 () {56 vars in 1180 bytes} [Fri May 27 15:00:48 2022] GET /identity/v3/groups/d49ca26b382043a0a226f9249c3326c8 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:49.005063 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2041f9-4b07-42f2-9926-033539bda88f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:49.024890 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3b5417-2c20-421a-94ee-dea026f306e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:49.025692 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b5417-2c20-421a-94ee-dea026f306e8 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:49.025979 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b5417-2c20-421a-94ee-dea026f306e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:49.026260 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b5417-2c20-421a-94ee-dea026f306e8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:49.039540 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3b5417-2c20-421a-94ee-dea026f306e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:49.048159 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-0f3b5417-2c20-421a-94ee-dea026f306e8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 15:00:49.048159 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:49.048159 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:49.048159 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:49.048159 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:49.048159 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:49.048159 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:49.048159 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:49.048159 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:49.048159 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:49.048159 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:49.048159 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:49.048159 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:49.048159 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:49.048159 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:49.048159 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:49.049076 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:49.049076 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:49.049076 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:49.049076 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:49.049076 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:49.049076 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:49.049076 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:49.049076 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:49.049076 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:49.049076 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:49.049076 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:49.049076 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:00:49.049076 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:49.049076 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:49.049076 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:49.049076 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:49.049875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:49.049875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:49.049875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:49.049875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:49.049875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:49.049875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:49.049875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:49.049875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:49.049875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:49.049875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:49.049875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 15:00:49.049875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:00:49.049875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 179/358] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:00:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:00:49.050548 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3b5417-2c20-421a-94ee-dea026f306e8 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:49.054974 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:49.111246 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:49.123991 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2041f9-4b07-42f2-9926-033539bda88f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:49.170412 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5a7c39-8d61-47c4-af33-385c6d9b35a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:49.171818 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a7c39-8d61-47c4-af33-385c6d9b35a0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:49.172253 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a7c39-8d61-47c4-af33-385c6d9b35a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:49.174301 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a7c39-8d61-47c4-af33-385c6d9b35a0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:49.186114 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3b5417-2c20-421a-94ee-dea026f306e8 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:49.191858 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5a7c39-8d61-47c4-af33-385c6d9b35a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:49.202358 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-7a5a7c39-8d61-47c4-af33-385c6d9b35a0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 15:00:49.202358 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:49.202358 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:49.202358 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:49.202358 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:49.202358 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:49.202358 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:49.202358 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:49.202358 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:49.202358 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:49.202358 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:49.202358 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:49.202358 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:49.202358 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:49.202358 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:49.202358 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:49.204100 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:49.204100 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:49.204100 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:49.204100 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:49.204100 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:49.204100 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:49.204100 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:49.204100 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:49.204100 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:49.204100 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:49.204100 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:49.204100 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:00:49.204100 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:49.204100 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:49.204100 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:49.204100 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:49.205096 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:49.205096 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:49.205096 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:49.205096 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:49.205096 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:49.205096 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:49.205096 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:49.205096 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:49.205096 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:49.205096 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:49.205096 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 15:00:49.205096 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:00:49.205096 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 180/359] 10.222.0.50 () {56 vars in 1114 bytes} [Fri May 27 15:00:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:00:49.205896 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5a7c39-8d61-47c4-af33-385c6d9b35a0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:49.235725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7598de47-a9ae-4d9e-927b-f06bb2bbb5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:49.237208 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598de47-a9ae-4d9e-927b-f06bb2bbb5a6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:49.237827 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598de47-a9ae-4d9e-927b-f06bb2bbb5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:49.239005 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598de47-a9ae-4d9e-927b-f06bb2bbb5a6 None admin] PATH_INFO: `/v3/users/ac3b33aa9c8b4888b81cf4eb021666b8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:49.240049 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7598de47-a9ae-4d9e-927b-f06bb2bbb5a6 None admin] RBAC: Authorizing `identity:get_user(user_id=ac3b33aa9c8b4888b81cf4eb021666b8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:49.241874 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7598de47-a9ae-4d9e-927b-f06bb2bbb5a6 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:49.243752 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 180/360] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:00:49 2022] GET /identity/v3/users/ac3b33aa9c8b4888b81cf4eb021666b8 => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:49.247118 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7598de47-a9ae-4d9e-927b-f06bb2bbb5a6 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:49.337060 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:49.342878 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5a7c39-8d61-47c4-af33-385c6d9b35a0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:49.381032 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7598de47-a9ae-4d9e-927b-f06bb2bbb5a6 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:49.390582 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb7923b-e9e0-4608-84ae-aaa7d3c7660e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:49.392048 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7923b-e9e0-4608-84ae-aaa7d3c7660e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:49.392447 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7923b-e9e0-4608-84ae-aaa7d3c7660e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:49.392726 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7923b-e9e0-4608-84ae-aaa7d3c7660e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:49.393992 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb7923b-e9e0-4608-84ae-aaa7d3c7660e None admin] RBAC: Adding query filter params (group.id=d49ca26b382043a0a226f9249c3326c8, role.id=ad9498a5e2064a468acc151fb2bce49e, scope.project.id=00203c7cc02341ee8ee7a591095f0c7d) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:49.393992 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb7923b-e9e0-4608-84ae-aaa7d3c7660e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:49.395903 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb7923b-e9e0-4608-84ae-aaa7d3c7660e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:49.400173 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:49.406681 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 181/361] 10.222.0.50 () {56 vars in 1376 bytes} [Fri May 27 15:00:49 2022] GET /identity/v3/role_assignments?group.id=d49ca26b382043a0a226f9249c3326c8&scope.project.id=00203c7cc02341ee8ee7a591095f0c7d&role.id=ad9498a5e2064a468acc151fb2bce49e => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:49.406681 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1f2fc-8dd2-49c5-8f85-b482452193d2 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:49.406681 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1f2fc-8dd2-49c5-8f85-b482452193d2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:49.406681 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1f2fc-8dd2-49c5-8f85-b482452193d2 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:49.406681 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 182/362] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:49.406681 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f1f2fc-8dd2-49c5-8f85-b482452193d2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:49.422057 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feda7f28-b954-435b-b413-7632bc10247b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:49.422968 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda7f28-b954-435b-b413-7632bc10247b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:49.423382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda7f28-b954-435b-b413-7632bc10247b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:49.423766 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feda7f28-b954-435b-b413-7632bc10247b None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:49.424786 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feda7f28-b954-435b-b413-7632bc10247b None admin] RBAC: Adding query filter params (name=member) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:49.425328 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feda7f28-b954-435b-b413-7632bc10247b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:49.426750 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feda7f28-b954-435b-b413-7632bc10247b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:49.438674 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 181/363] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 15:00:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:49.439742 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feda7f28-b954-435b-b413-7632bc10247b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:49.548590 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f1f2fc-8dd2-49c5-8f85-b482452193d2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:49.559180 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feda7f28-b954-435b-b413-7632bc10247b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:49.596606 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dce15cc-a688-470c-93e6-65c2c479a396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:49.597659 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dce15cc-a688-470c-93e6-65c2c479a396 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:49.598043 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dce15cc-a688-470c-93e6-65c2c479a396 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:49.598419 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dce15cc-a688-470c-93e6-65c2c479a396 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:49.599332 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54492203-00da-448a-bd81-9ce74bf8a563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:49.599537 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dce15cc-a688-470c-93e6-65c2c479a396 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:49.599949 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dce15cc-a688-470c-93e6-65c2c479a396 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:49.600051 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54492203-00da-448a-bd81-9ce74bf8a563 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:49.600310 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54492203-00da-448a-bd81-9ce74bf8a563 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:49.600637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54492203-00da-448a-bd81-9ce74bf8a563 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:49.601468 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54492203-00da-448a-bd81-9ce74bf8a563 None admin] RBAC: Adding query filter params (role.id=1b313eb2327b44b7a11d92914db65bb9, user.id=ac3b33aa9c8b4888b81cf4eb021666b8, scope.system=all) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:49.601795 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dce15cc-a688-470c-93e6-65c2c479a396 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:49.601897 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54492203-00da-448a-bd81-9ce74bf8a563 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:49.603409 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54492203-00da-448a-bd81-9ce74bf8a563 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:49.611714 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 183/364] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:00:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:49.617762 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 182/365] 10.222.0.50 () {56 vars in 1308 bytes} [Fri May 27 15:00:49 2022] GET /identity/v3/role_assignments?user.id=ac3b33aa9c8b4888b81cf4eb021666b8&scope.system=all&role.id=1b313eb2327b44b7a11d92914db65bb9 => generated 510 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:49.617959 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0b08c-bbea-4e90-a5d3-814b8b91c38d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:49.618156 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0b08c-bbea-4e90-a5d3-814b8b91c38d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:49.618435 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0b08c-bbea-4e90-a5d3-814b8b91c38d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:49.618611 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54492203-00da-448a-bd81-9ce74bf8a563 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:49.644572 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3bd0b08c-bbea-4e90-a5d3-814b8b91c38d None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:49.645909 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3bd0b08c-bbea-4e90-a5d3-814b8b91c38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:49.669412 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:49.682456 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:49.737414 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 184/366] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:00:49.738377 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0b08c-bbea-4e90-a5d3-814b8b91c38d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:49.827881 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54492203-00da-448a-bd81-9ce74bf8a563 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:49.856762 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0b08c-bbea-4e90-a5d3-814b8b91c38d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:49.874663 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78df5a70-5534-496a-bea3-847a2ffbc011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:49.876528 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df5a70-5534-496a-bea3-847a2ffbc011 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:49.877064 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df5a70-5534-496a-bea3-847a2ffbc011 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:49.877548 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78df5a70-5534-496a-bea3-847a2ffbc011 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:49.896872 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e1552d-c464-47b3-ae49-cf8aeabbff7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:49.897542 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e1552d-c464-47b3-ae49-cf8aeabbff7b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:49.897824 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e1552d-c464-47b3-ae49-cf8aeabbff7b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:49.898100 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e1552d-c464-47b3-ae49-cf8aeabbff7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:49.927595 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:49.956724 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78df5a70-5534-496a-bea3-847a2ffbc011 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:49.958564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78df5a70-5534-496a-bea3-847a2ffbc011 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:49.977048 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e1552d-c464-47b3-ae49-cf8aeabbff7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:49.978535 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e1552d-c464-47b3-ae49-cf8aeabbff7b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:50.007230 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:50.073139 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 183/367] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:00:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 15:00:50.074398 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78df5a70-5534-496a-bea3-847a2ffbc011 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:50.099663 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 185/368] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:00:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 15:00:50.100483 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e1552d-c464-47b3-ae49-cf8aeabbff7b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:50.167304 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:50.192581 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78df5a70-5534-496a-bea3-847a2ffbc011 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:50.227287 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e1552d-c464-47b3-ae49-cf8aeabbff7b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:50.237775 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d31a22-2014-49ec-aa88-85fb0eed5b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:50.238573 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d31a22-2014-49ec-aa88-85fb0eed5b45 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:50.238867 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d31a22-2014-49ec-aa88-85fb0eed5b45 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:50.239157 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d31a22-2014-49ec-aa88-85fb0eed5b45 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:50.247049 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:50.253231 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d31a22-2014-49ec-aa88-85fb0eed5b45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:50.261456 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-f4d31a22-2014-49ec-aa88-85fb0eed5b45 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:00:50.261456 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:50.261456 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:50.261456 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:50.261456 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:50.261456 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:50.261456 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:50.261456 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:50.261456 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:50.261456 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:50.261456 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:50.261456 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:50.261456 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:50.261456 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:50.261456 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:50.261456 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:50.262341 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:50.262341 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:50.262341 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:50.262341 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:50.262341 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:50.262341 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:50.262341 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:50.262341 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:50.262341 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:50.262341 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:50.262341 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:50.262341 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:00:50.262341 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:50.262341 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:50.262341 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:50.262341 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:50.263165 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:50.263165 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:50.263165 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:50.263165 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:50.263165 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:50.263165 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:50.263165 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:50.263165 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:50.263165 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:50.263165 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:50.263165 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:00:50.263165 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:00:50.263165 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 184/369] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 15:00:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:00:50.264403 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d31a22-2014-49ec-aa88-85fb0eed5b45 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:50.279587 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd36eea7-cd9b-4801-adbf-f35cbdfe09f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:50.280462 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36eea7-cd9b-4801-adbf-f35cbdfe09f8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:50.280747 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36eea7-cd9b-4801-adbf-f35cbdfe09f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:50.281025 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36eea7-cd9b-4801-adbf-f35cbdfe09f8 None admin] PATH_INFO: `/v3/projects/65801518f2424895ab7af78ae7acb341` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:50.283547 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd36eea7-cd9b-4801-adbf-f35cbdfe09f8 None admin] RBAC: Authorizing `identity:get_project(project_id=65801518f2424895ab7af78ae7acb341)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:50.284988 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd36eea7-cd9b-4801-adbf-f35cbdfe09f8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:50.287254 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 186/370] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:00:50 2022] GET /identity/v3/projects/65801518f2424895ab7af78ae7acb341 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:50.288036 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd36eea7-cd9b-4801-adbf-f35cbdfe09f8 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:50.398182 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d31a22-2014-49ec-aa88-85fb0eed5b45 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:50.414950 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd36eea7-cd9b-4801-adbf-f35cbdfe09f8 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:50.436977 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60b8029-2db4-47ad-b49a-01b1a80289f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:50.438333 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b8029-2db4-47ad-b49a-01b1a80289f9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:50.438618 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b8029-2db4-47ad-b49a-01b1a80289f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:50.439411 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b8029-2db4-47ad-b49a-01b1a80289f9 None admin] PATH_INFO: `/v3/projects/65801518f2424895ab7af78ae7acb341` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:50.440724 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60b8029-2db4-47ad-b49a-01b1a80289f9 None admin] RBAC: Authorizing `identity:get_project(project_id=65801518f2424895ab7af78ae7acb341)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:50.442316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60b8029-2db4-47ad-b49a-01b1a80289f9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:50.444119 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:50.445176 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 185/371] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:00:50 2022] GET /identity/v3/projects/65801518f2424895ab7af78ae7acb341 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:50.445961 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60b8029-2db4-47ad-b49a-01b1a80289f9 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:50.465615 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbef121-e0f5-4b39-a608-4e16a820b75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:50.466294 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbef121-e0f5-4b39-a608-4e16a820b75a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:50.466830 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbef121-e0f5-4b39-a608-4e16a820b75a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:50.467134 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbef121-e0f5-4b39-a608-4e16a820b75a None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:50.468287 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbef121-e0f5-4b39-a608-4e16a820b75a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:50.468594 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbef121-e0f5-4b39-a608-4e16a820b75a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:50.470623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbef121-e0f5-4b39-a608-4e16a820b75a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:50.483333 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 187/372] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:00:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:50.484266 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbef121-e0f5-4b39-a608-4e16a820b75a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:50.522299 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:50.563916 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60b8029-2db4-47ad-b49a-01b1a80289f9 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:50.606740 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373dbed0-7a0c-467f-abb6-95b7fae42867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:50.607811 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373dbed0-7a0c-467f-abb6-95b7fae42867 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:50.608294 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373dbed0-7a0c-467f-abb6-95b7fae42867 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:50.608768 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373dbed0-7a0c-467f-abb6-95b7fae42867 None admin] PATH_INFO: `/v3/groups/d49ca26b382043a0a226f9249c3326c8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:50.610227 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373dbed0-7a0c-467f-abb6-95b7fae42867 None admin] RBAC: Authorizing `identity:get_group(group_id=d49ca26b382043a0a226f9249c3326c8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:50.611636 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbef121-e0f5-4b39-a608-4e16a820b75a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:50.611872 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373dbed0-7a0c-467f-abb6-95b7fae42867 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:50.613966 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 186/373] 10.222.0.50 () {56 vars in 1180 bytes} [Fri May 27 15:00:50 2022] GET /identity/v3/groups/d49ca26b382043a0a226f9249c3326c8 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:50.615005 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373dbed0-7a0c-467f-abb6-95b7fae42867 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:50.655703 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8173d35f-9164-4435-afdb-bc01fa66002a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:50.657254 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8173d35f-9164-4435-afdb-bc01fa66002a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:50.657543 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8173d35f-9164-4435-afdb-bc01fa66002a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:50.657820 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8173d35f-9164-4435-afdb-bc01fa66002a None admin] PATH_INFO: `/v3/groups/d49ca26b382043a0a226f9249c3326c8` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:50.659079 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8173d35f-9164-4435-afdb-bc01fa66002a None admin] RBAC: Authorizing `identity:get_group(group_id=d49ca26b382043a0a226f9249c3326c8)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:50.661617 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8173d35f-9164-4435-afdb-bc01fa66002a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:50.663619 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 188/374] 10.222.0.50 () {56 vars in 1180 bytes} [Fri May 27 15:00:50 2022] GET /identity/v3/groups/d49ca26b382043a0a226f9249c3326c8 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:50.667051 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8173d35f-9164-4435-afdb-bc01fa66002a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:50.733292 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:50.743545 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373dbed0-7a0c-467f-abb6-95b7fae42867 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:50.795365 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0151d76-7f27-4ac9-a30c-2608706443b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:50.796258 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0151d76-7f27-4ac9-a30c-2608706443b8 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:50.796708 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0151d76-7f27-4ac9-a30c-2608706443b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:50.797164 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0151d76-7f27-4ac9-a30c-2608706443b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:50.800600 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8173d35f-9164-4435-afdb-bc01fa66002a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:50.800820 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:50.845055 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc01daff-5283-4e20-b1f4-5f5ec42b1e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:50.846074 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01daff-5283-4e20-b1f4-5f5ec42b1e80 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:50.846569 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01daff-5283-4e20-b1f4-5f5ec42b1e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:50.847140 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc01daff-5283-4e20-b1f4-5f5ec42b1e80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:50.848329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc01daff-5283-4e20-b1f4-5f5ec42b1e80 None admin] RBAC: Adding query filter params (group.id=d49ca26b382043a0a226f9249c3326c8, role.id=8442c42e847a4b38b76f3a7fbe9ada85, scope.project.id=65801518f2424895ab7af78ae7acb341) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:50.848841 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc01daff-5283-4e20-b1f4-5f5ec42b1e80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:50.850709 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc01daff-5283-4e20-b1f4-5f5ec42b1e80 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:50.857428 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 189/375] 10.222.0.50 () {56 vars in 1376 bytes} [Fri May 27 15:00:50 2022] GET /identity/v3/role_assignments?group.id=d49ca26b382043a0a226f9249c3326c8&scope.project.id=65801518f2424895ab7af78ae7acb341&role.id=8442c42e847a4b38b76f3a7fbe9ada85 => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:50.861348 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc01daff-5283-4e20-b1f4-5f5ec42b1e80 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:50.887653 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0151d76-7f27-4ac9-a30c-2608706443b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:50.889510 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0151d76-7f27-4ac9-a30c-2608706443b8 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:51.026266 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 187/376] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:00:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 15:00:51.031601 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0151d76-7f27-4ac9-a30c-2608706443b8 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:51.062415 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc01daff-5283-4e20-b1f4-5f5ec42b1e80 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:51.079247 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:51.108817 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:51.159744 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0151d76-7f27-4ac9-a30c-2608706443b8 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:51.161798 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d898623-f97f-4f8e-b03b-f036df8e1101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:51.162472 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d898623-f97f-4f8e-b03b-f036df8e1101 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:51.162809 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d898623-f97f-4f8e-b03b-f036df8e1101 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:51.164034 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d898623-f97f-4f8e-b03b-f036df8e1101 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:51.164878 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d898623-f97f-4f8e-b03b-f036df8e1101 None admin] RBAC: Adding query filter params (group.id=d49ca26b382043a0a226f9249c3326c8, role.id=ad9498a5e2064a468acc151fb2bce49e, scope.project.id=65801518f2424895ab7af78ae7acb341) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:51.165190 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d898623-f97f-4f8e-b03b-f036df8e1101 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:51.167568 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d898623-f97f-4f8e-b03b-f036df8e1101 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:51.174725 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 190/377] 10.222.0.50 () {56 vars in 1376 bytes} [Fri May 27 15:00:50 2022] GET /identity/v3/role_assignments?group.id=d49ca26b382043a0a226f9249c3326c8&scope.project.id=65801518f2424895ab7af78ae7acb341&role.id=ad9498a5e2064a468acc151fb2bce49e => generated 260 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:51.176412 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e35ae02-c076-48b0-849b-6f1d62d47fb3 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:51.176698 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e35ae02-c076-48b0-849b-6f1d62d47fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:51.176973 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e35ae02-c076-48b0-849b-6f1d62d47fb3 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:51.178814 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 191/378] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:51.184600 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbefe4d9-c4a5-4ef2-bf48-c2db37fb1ce6 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:51.184901 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbefe4d9-c4a5-4ef2-bf48-c2db37fb1ce6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:51.185211 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbefe4d9-c4a5-4ef2-bf48-c2db37fb1ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:51.206916 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-cbefe4d9-c4a5-4ef2-bf48-c2db37fb1ce6 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:51.206916 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cbefe4d9-c4a5-4ef2-bf48-c2db37fb1ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:51.216685 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e735d-dadf-42ea-87e5-c1cc13e36270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:51.217566 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e735d-dadf-42ea-87e5-c1cc13e36270 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:51.217855 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e735d-dadf-42ea-87e5-c1cc13e36270 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:51.218133 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e735d-dadf-42ea-87e5-c1cc13e36270 None admin] PATH_INFO: `/v3/projects/6fdd45226b7e4d17bd9b93b1a5b70196` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:51.220110 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033e735d-dadf-42ea-87e5-c1cc13e36270 None admin] RBAC: Authorizing `identity:get_project(project_id=6fdd45226b7e4d17bd9b93b1a5b70196)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:51.221717 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033e735d-dadf-42ea-87e5-c1cc13e36270 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:51.226941 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 188/379] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:00:51 2022] GET /identity/v3/projects/6fdd45226b7e4d17bd9b93b1a5b70196 => generated 344 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:51.231377 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e735d-dadf-42ea-87e5-c1cc13e36270 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:51.325012 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 192/380] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 15:00:51.365507 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbefe4d9-c4a5-4ef2-bf48-c2db37fb1ce6 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:51.428448 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e735d-dadf-42ea-87e5-c1cc13e36270 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:51.434760 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:51.495303 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e4f4a-0c6e-4fff-82d3-3b51cbfb20bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:51.496527 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e4f4a-0c6e-4fff-82d3-3b51cbfb20bf None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:51.497077 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e4f4a-0c6e-4fff-82d3-3b51cbfb20bf None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:51.497960 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027e4f4a-0c6e-4fff-82d3-3b51cbfb20bf None admin] PATH_INFO: `/v3/groups/90f7c40ee18847979e4a1a72530ac3ec` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:51.504706 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027e4f4a-0c6e-4fff-82d3-3b51cbfb20bf None admin] RBAC: Authorizing `identity:get_group(group_id=90f7c40ee18847979e4a1a72530ac3ec)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:51.506785 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027e4f4a-0c6e-4fff-82d3-3b51cbfb20bf None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:51.509547 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 189/381] 10.222.0.50 () {56 vars in 1180 bytes} [Fri May 27 15:00:51 2022] GET /identity/v3/groups/90f7c40ee18847979e4a1a72530ac3ec => generated 229 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:51.518451 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e4f4a-0c6e-4fff-82d3-3b51cbfb20bf None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:51.533558 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbefe4d9-c4a5-4ef2-bf48-c2db37fb1ce6 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:51.546012 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:51.595225 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b08ba05-7cb7-4873-ab2e-2f4da426a808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:51.595953 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08ba05-7cb7-4873-ab2e-2f4da426a808 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:51.596234 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08ba05-7cb7-4873-ab2e-2f4da426a808 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:51.596504 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08ba05-7cb7-4873-ab2e-2f4da426a808 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:51.619837 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b08ba05-7cb7-4873-ab2e-2f4da426a808 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:51.628402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-8b08ba05-7cb7-4873-ab2e-2f4da426a808 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 15:00:51.628402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:51.628402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:51.628402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:51.628402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:51.628402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:51.628402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:51.628402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:51.628402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:51.628402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:51.628402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:51.628402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:51.628402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:51.628402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:51.628402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:51.628402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:51.629329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:51.629329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:51.629329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:51.629329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:51.629329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:51.629329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:51.629329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:51.629329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:51.629329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:51.629329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:51.629329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:51.629329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:00:51.629329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:51.629329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:51.629329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:51.629329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:51.630134 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:51.630134 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:51.630134 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:51.630134 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:51.630134 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:51.630134 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:51.630134 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:51.630134 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:51.630134 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:51.630134 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:51.630134 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 15:00:51.630134 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:00:51.635586 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 193/382] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:00:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:00:51.643417 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b08ba05-7cb7-4873-ab2e-2f4da426a808 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:51.691604 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027e4f4a-0c6e-4fff-82d3-3b51cbfb20bf None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:51.737575 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136262c1-395c-490e-b4a1-3a268c4fd2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:51.738315 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136262c1-395c-490e-b4a1-3a268c4fd2f9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:51.738603 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136262c1-395c-490e-b4a1-3a268c4fd2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:51.738900 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136262c1-395c-490e-b4a1-3a268c4fd2f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:51.742876 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136262c1-395c-490e-b4a1-3a268c4fd2f9 None admin] RBAC: Adding query filter params (group.id=90f7c40ee18847979e4a1a72530ac3ec, role.id=759d845eb6284e42931b4265af448ebc, scope.project.id=6fdd45226b7e4d17bd9b93b1a5b70196) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:51.742876 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136262c1-395c-490e-b4a1-3a268c4fd2f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:51.742876 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136262c1-395c-490e-b4a1-3a268c4fd2f9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:51.749037 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 190/383] 10.222.0.50 () {56 vars in 1376 bytes} [Fri May 27 15:00:51 2022] GET /identity/v3/role_assignments?group.id=90f7c40ee18847979e4a1a72530ac3ec&scope.project.id=6fdd45226b7e4d17bd9b93b1a5b70196&role.id=759d845eb6284e42931b4265af448ebc => generated 260 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:51.827559 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b08ba05-7cb7-4873-ab2e-2f4da426a808 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:51.867526 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d4608-35b7-4020-b520-fd0ca3a3a092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:51.868561 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d4608-35b7-4020-b520-fd0ca3a3a092 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:51.869051 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d4608-35b7-4020-b520-fd0ca3a3a092 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:51.869555 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927d4608-35b7-4020-b520-fd0ca3a3a092 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:51.870526 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d4608-35b7-4020-b520-fd0ca3a3a092 None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:51.871068 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d4608-35b7-4020-b520-fd0ca3a3a092 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:51.872682 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927d4608-35b7-4020-b520-fd0ca3a3a092 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:51.884566 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 194/384] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 15:00:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:51.891582 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136262c1-395c-490e-b4a1-3a268c4fd2f9 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:51.945000 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:52.034588 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136262c1-395c-490e-b4a1-3a268c4fd2f9 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:52.087225 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9598ce-556f-4546-ad5e-4f4d77e76c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:52.088409 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9598ce-556f-4546-ad5e-4f4d77e76c6c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:52.088694 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9598ce-556f-4546-ad5e-4f4d77e76c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:52.089730 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9598ce-556f-4546-ad5e-4f4d77e76c6c None admin] PATH_INFO: `/v3/groups/d49ca26b382043a0a226f9249c3326c8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:52.091144 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9598ce-556f-4546-ad5e-4f4d77e76c6c None admin] RBAC: Authorizing `identity:get_group(group_id=d49ca26b382043a0a226f9249c3326c8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:52.093245 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9598ce-556f-4546-ad5e-4f4d77e76c6c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:52.095428 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 191/385] 10.222.0.50 () {56 vars in 1180 bytes} [Fri May 27 15:00:51 2022] GET /identity/v3/groups/d49ca26b382043a0a226f9249c3326c8 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:52.100815 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d4608-35b7-4020-b520-fd0ca3a3a092 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:52.143741 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:52.230835 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927d4608-35b7-4020-b520-fd0ca3a3a092 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:52.266258 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a738f69-67de-4b23-a494-0f3d5ff52d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:52.267161 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a738f69-67de-4b23-a494-0f3d5ff52d1b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:52.267443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a738f69-67de-4b23-a494-0f3d5ff52d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:52.267718 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a738f69-67de-4b23-a494-0f3d5ff52d1b None admin] PATH_INFO: `/v3/projects/00203c7cc02341ee8ee7a591095f0c7d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:52.268920 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a738f69-67de-4b23-a494-0f3d5ff52d1b None admin] RBAC: Authorizing `identity:get_project(project_id=00203c7cc02341ee8ee7a591095f0c7d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:52.270360 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a738f69-67de-4b23-a494-0f3d5ff52d1b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:52.272983 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 195/386] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:00:52 2022] GET /identity/v3/projects/00203c7cc02341ee8ee7a591095f0c7d => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:52.278267 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9598ce-556f-4546-ad5e-4f4d77e76c6c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:52.413004 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50fe288-fb2e-49ea-baf0-8a2b6f0eca1a None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:52.413613 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50fe288-fb2e-49ea-baf0-8a2b6f0eca1a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:52.414092 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50fe288-fb2e-49ea-baf0-8a2b6f0eca1a None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:52.415701 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 196/387] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 15:00:52.420963 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4637b9e-4ed3-4bc8-98b0-53e93cf29292 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:52.421251 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4637b9e-4ed3-4bc8-98b0-53e93cf29292 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:52.421528 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4637b9e-4ed3-4bc8-98b0-53e93cf29292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:52.441234 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:52.445136 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9598ce-556f-4546-ad5e-4f4d77e76c6c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:52.447630 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f4637b9e-4ed3-4bc8-98b0-53e93cf29292 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:52.449033 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f4637b9e-4ed3-4bc8-98b0-53e93cf29292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:52.535601 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388e7cfa-4333-4b5a-a29c-c156d4b416a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:52.536491 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e7cfa-4333-4b5a-a29c-c156d4b416a4 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:52.536781 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e7cfa-4333-4b5a-a29c-c156d4b416a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:52.537005 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 197/388] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:00:52.537115 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e7cfa-4333-4b5a-a29c-c156d4b416a4 None admin] PATH_INFO: `/v3/projects/00203c7cc02341ee8ee7a591095f0c7d/groups/d49ca26b382043a0a226f9249c3326c8/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:52.543673 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388e7cfa-4333-4b5a-a29c-c156d4b416a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=00203c7cc02341ee8ee7a591095f0c7d, group_id=d49ca26b382043a0a226f9249c3326c8, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:52.545149 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388e7cfa-4333-4b5a-a29c-c156d4b416a4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:52.559559 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4637b9e-4ed3-4bc8-98b0-53e93cf29292 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:52.575084 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 192/389] 10.222.0.50 () {58 vars in 1442 bytes} [Fri May 27 15:00:52 2022] PUT /identity/v3/projects/00203c7cc02341ee8ee7a591095f0c7d/groups/d49ca26b382043a0a226f9249c3326c8/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:00:52.599352 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:52.686184 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4637b9e-4ed3-4bc8-98b0-53e93cf29292 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:52.727904 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50638e20-c5f5-492c-9742-1928429c1ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:52.728721 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50638e20-c5f5-492c-9742-1928429c1ec1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:52.729000 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50638e20-c5f5-492c-9742-1928429c1ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:52.729269 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50638e20-c5f5-492c-9742-1928429c1ec1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:52.744110 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50638e20-c5f5-492c-9742-1928429c1ec1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:52.752803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-50638e20-c5f5-492c-9742-1928429c1ec1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 15:00:52.752803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:52.752803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:52.752803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:52.752803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:52.752803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:52.752803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:52.752803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:52.752803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:52.752803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:52.752803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:52.752803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:52.752803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:52.752803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:52.752803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:52.752803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:52.753749 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:52.753749 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:52.753749 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:52.753749 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:52.753749 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:52.753749 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:52.753749 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:52.753749 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:52.753749 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:52.753749 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:52.753749 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:52.753749 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:00:52.753749 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:52.753749 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:52.753749 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:52.753749 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:52.754794 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:52.754794 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:52.754794 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:52.754794 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:52.754794 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:52.754794 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:52.754794 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:52.754794 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:52.754794 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:52.754794 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:52.754794 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 15:00:52.754794 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:00:52.754794 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 198/390] 10.222.0.50 () {56 vars in 1114 bytes} [Fri May 27 15:00:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:00:52.760031 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388e7cfa-4333-4b5a-a29c-c156d4b416a4 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:52.919695 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388e7cfa-4333-4b5a-a29c-c156d4b416a4 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:52.963854 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c74a2-04d3-4983-b80d-cfc11dcf1d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:52.964455 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c74a2-04d3-4983-b80d-cfc11dcf1d38 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:52.964742 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c74a2-04d3-4983-b80d-cfc11dcf1d38 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:52.965027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c74a2-04d3-4983-b80d-cfc11dcf1d38 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:52.965781 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c74a2-04d3-4983-b80d-cfc11dcf1d38 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:52.966089 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c74a2-04d3-4983-b80d-cfc11dcf1d38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:52.967501 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c74a2-04d3-4983-b80d-cfc11dcf1d38 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:52.983051 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 193/391] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:00:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:52.988596 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50638e20-c5f5-492c-9742-1928429c1ec1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:53.107177 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:53.118355 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50638e20-c5f5-492c-9742-1928429c1ec1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:53.178824 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55761fc-c6dc-470b-8e5a-9abde42bfd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:53.179713 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55761fc-c6dc-470b-8e5a-9abde42bfd48 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:53.179997 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55761fc-c6dc-470b-8e5a-9abde42bfd48 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:53.180273 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55761fc-c6dc-470b-8e5a-9abde42bfd48 None admin] PATH_INFO: `/v3/groups/d49ca26b382043a0a226f9249c3326c8` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:53.188130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55761fc-c6dc-470b-8e5a-9abde42bfd48 None admin] RBAC: Authorizing `identity:get_group(group_id=d49ca26b382043a0a226f9249c3326c8)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:53.189723 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55761fc-c6dc-470b-8e5a-9abde42bfd48 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:53.191446 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 199/392] 10.222.0.50 () {56 vars in 1180 bytes} [Fri May 27 15:00:52 2022] GET /identity/v3/groups/d49ca26b382043a0a226f9249c3326c8 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:53.197046 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c74a2-04d3-4983-b80d-cfc11dcf1d38 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:53.225909 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:53.332713 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c74a2-04d3-4983-b80d-cfc11dcf1d38 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:53.379570 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324259f9-55c7-4a82-a9b3-67f65f150ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:53.380456 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324259f9-55c7-4a82-a9b3-67f65f150ac9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:53.380742 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324259f9-55c7-4a82-a9b3-67f65f150ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:53.381017 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324259f9-55c7-4a82-a9b3-67f65f150ac9 None admin] PATH_INFO: `/v3/projects/00203c7cc02341ee8ee7a591095f0c7d` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:53.382243 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324259f9-55c7-4a82-a9b3-67f65f150ac9 None admin] RBAC: Authorizing `identity:get_project(project_id=00203c7cc02341ee8ee7a591095f0c7d)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:53.383709 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324259f9-55c7-4a82-a9b3-67f65f150ac9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:53.385840 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 194/393] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:00:53 2022] GET /identity/v3/projects/00203c7cc02341ee8ee7a591095f0c7d => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:53.391545 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55761fc-c6dc-470b-8e5a-9abde42bfd48 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:53.521962 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114aa4f-8711-4af5-b410-724600e21b82 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:53.522257 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114aa4f-8711-4af5-b410-724600e21b82 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:53.522527 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114aa4f-8711-4af5-b410-724600e21b82 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:53.524086 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 195/394] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 15:00:53.527459 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55761fc-c6dc-470b-8e5a-9abde42bfd48 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:53.534113 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1c626-3b15-4264-8ac0-a8c94127e7cd None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:53.534695 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1c626-3b15-4264-8ac0-a8c94127e7cd None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:53.535196 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca1c626-3b15-4264-8ac0-a8c94127e7cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:53.575770 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1283a1-824f-4401-88fa-4d0412ba10b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:53.576219 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4ca1c626-3b15-4264-8ac0-a8c94127e7cd None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:53.576664 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1283a1-824f-4401-88fa-4d0412ba10b0 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:53.576952 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1283a1-824f-4401-88fa-4d0412ba10b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:53.577232 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1283a1-824f-4401-88fa-4d0412ba10b0 None admin] PATH_INFO: `/v3/projects/00203c7cc02341ee8ee7a591095f0c7d/groups/d49ca26b382043a0a226f9249c3326c8/roles/ad9498a5e2064a468acc151fb2bce49e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:53.578262 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4ca1c626-3b15-4264-8ac0-a8c94127e7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:53.582875 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1283a1-824f-4401-88fa-4d0412ba10b0 None admin] RBAC: Authorizing `identity:create_grant(project_id=00203c7cc02341ee8ee7a591095f0c7d, group_id=d49ca26b382043a0a226f9249c3326c8, role_id=ad9498a5e2064a468acc151fb2bce49e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:53.582875 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1283a1-824f-4401-88fa-4d0412ba10b0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:53.605423 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 200/395] 10.222.0.50 () {58 vars in 1442 bytes} [Fri May 27 15:00:53 2022] PUT /identity/v3/projects/00203c7cc02341ee8ee7a591095f0c7d/groups/d49ca26b382043a0a226f9249c3326c8/roles/ad9498a5e2064a468acc151fb2bce49e => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:00:53.695215 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 196/396] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 15:00:53.730307 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1283a1-824f-4401-88fa-4d0412ba10b0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:53.760752 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:53.873080 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1283a1-824f-4401-88fa-4d0412ba10b0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:53.909502 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa6af5e-7081-4109-bf92-e50b3ba6f46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:53.910314 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6af5e-7081-4109-bf92-e50b3ba6f46a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:53.910601 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6af5e-7081-4109-bf92-e50b3ba6f46a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:53.910903 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6af5e-7081-4109-bf92-e50b3ba6f46a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:53.922650 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa6af5e-7081-4109-bf92-e50b3ba6f46a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:53.930685 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-5fa6af5e-7081-4109-bf92-e50b3ba6f46a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 15:00:53.930685 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:53.930685 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:53.930685 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:53.930685 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:53.930685 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:53.930685 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:53.930685 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:53.930685 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:53.930685 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:53.930685 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:53.930685 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:53.930685 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:53.930685 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:53.930685 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:53.930685 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:53.931696 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:53.931696 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:53.931696 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:53.931696 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:53.931696 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:53.931696 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:53.931696 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:53.931696 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:53.931696 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:53.931696 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:53.931696 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:53.931696 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:00:53.931696 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:53.931696 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:53.931696 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:53.931696 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:53.932525 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:53.932525 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:53.932525 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:53.932525 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:53.932525 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:53.932525 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:53.932525 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:53.932525 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:53.932525 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:53.932525 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:53.932525 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 15:00:53.932525 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:00:53.932525 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 201/397] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:00:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:00:53.944057 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca1c626-3b15-4264-8ac0-a8c94127e7cd None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:53.969286 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:54.063814 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca1c626-3b15-4264-8ac0-a8c94127e7cd None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:54.103232 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9610bc-db48-4dc9-b05c-bbf35a79a157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:54.103923 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9610bc-db48-4dc9-b05c-bbf35a79a157 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:54.104206 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9610bc-db48-4dc9-b05c-bbf35a79a157 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:54.104481 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9610bc-db48-4dc9-b05c-bbf35a79a157 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:54.105243 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9610bc-db48-4dc9-b05c-bbf35a79a157 None admin] RBAC: Adding query filter params (name=member) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:54.105548 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9610bc-db48-4dc9-b05c-bbf35a79a157 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:54.107051 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9610bc-db48-4dc9-b05c-bbf35a79a157 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:54.117600 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 197/398] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 15:00:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:54.123247 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa6af5e-7081-4109-bf92-e50b3ba6f46a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:54.220598 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:54.246348 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa6af5e-7081-4109-bf92-e50b3ba6f46a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:54.289699 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f039d013-b4e1-4f92-80ea-36df5befc03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:54.290551 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039d013-b4e1-4f92-80ea-36df5befc03a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:54.290890 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039d013-b4e1-4f92-80ea-36df5befc03a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:54.291170 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f039d013-b4e1-4f92-80ea-36df5befc03a None admin] PATH_INFO: `/v3/groups/d49ca26b382043a0a226f9249c3326c8` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:54.295234 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f039d013-b4e1-4f92-80ea-36df5befc03a None admin] RBAC: Authorizing `identity:get_group(group_id=d49ca26b382043a0a226f9249c3326c8)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:54.296688 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f039d013-b4e1-4f92-80ea-36df5befc03a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:54.298444 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 202/399] 10.222.0.50 () {56 vars in 1180 bytes} [Fri May 27 15:00:54 2022] GET /identity/v3/groups/d49ca26b382043a0a226f9249c3326c8 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:54.303859 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9610bc-db48-4dc9-b05c-bbf35a79a157 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:54.399787 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:54.430817 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9610bc-db48-4dc9-b05c-bbf35a79a157 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:54.489318 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6fd9aa-21bd-4875-b46a-787ee8f303a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:54.490795 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6fd9aa-21bd-4875-b46a-787ee8f303a7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:54.491327 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6fd9aa-21bd-4875-b46a-787ee8f303a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:54.491822 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6fd9aa-21bd-4875-b46a-787ee8f303a7 None admin] PATH_INFO: `/v3/projects/65801518f2424895ab7af78ae7acb341` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:54.493850 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6fd9aa-21bd-4875-b46a-787ee8f303a7 None admin] RBAC: Authorizing `identity:get_project(project_id=65801518f2424895ab7af78ae7acb341)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:54.496599 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6fd9aa-21bd-4875-b46a-787ee8f303a7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:54.500189 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 198/400] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:00:54 2022] GET /identity/v3/projects/65801518f2424895ab7af78ae7acb341 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:54.509009 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f039d013-b4e1-4f92-80ea-36df5befc03a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:54.593746 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef51b91-1cc9-4832-9388-1af1c1c06f3d None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:54.594256 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef51b91-1cc9-4832-9388-1af1c1c06f3d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:54.594745 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef51b91-1cc9-4832-9388-1af1c1c06f3d None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:54.597213 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 199/401] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:54.612842 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9baf351-9c47-4e59-8dc8-a0b9e022a2c6 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:54.613333 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9baf351-9c47-4e59-8dc8-a0b9e022a2c6 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:54.613816 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9baf351-9c47-4e59-8dc8-a0b9e022a2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:54.661022 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a9baf351-9c47-4e59-8dc8-a0b9e022a2c6 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:54.662959 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a9baf351-9c47-4e59-8dc8-a0b9e022a2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:54.778985 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f039d013-b4e1-4f92-80ea-36df5befc03a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:54.805912 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 200/402] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 15:00:54.824566 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b957ddd-60fe-4e55-a765-5af8aed60e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:54.825444 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b957ddd-60fe-4e55-a765-5af8aed60e8e None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:54.825722 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b957ddd-60fe-4e55-a765-5af8aed60e8e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:54.826005 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b957ddd-60fe-4e55-a765-5af8aed60e8e None admin] PATH_INFO: `/v3/projects/65801518f2424895ab7af78ae7acb341/groups/d49ca26b382043a0a226f9249c3326c8/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:54.828068 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b957ddd-60fe-4e55-a765-5af8aed60e8e None admin] RBAC: Authorizing `identity:create_grant(project_id=65801518f2424895ab7af78ae7acb341, group_id=d49ca26b382043a0a226f9249c3326c8, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:54.830028 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b957ddd-60fe-4e55-a765-5af8aed60e8e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:54.849529 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9baf351-9c47-4e59-8dc8-a0b9e022a2c6 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:54.862112 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 203/403] 10.222.0.50 () {58 vars in 1442 bytes} [Fri May 27 15:00:54 2022] PUT /identity/v3/projects/65801518f2424895ab7af78ae7acb341/groups/d49ca26b382043a0a226f9249c3326c8/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:00:54.943422 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:54.967890 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9baf351-9c47-4e59-8dc8-a0b9e022a2c6 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:54.983042 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e4bfc4-f316-47f1-afdc-109becb53855 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:54.983377 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e4bfc4-f316-47f1-afdc-109becb53855 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:54.983607 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e4bfc4-f316-47f1-afdc-109becb53855 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:54.990422 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 204/404] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 15:00:54.996450 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a7db8-2fe5-4f21-807b-49cd67dba225 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:54.996774 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a7db8-2fe5-4f21-807b-49cd67dba225 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:54.997012 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a7db8-2fe5-4f21-807b-49cd67dba225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:55.015718 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:55.017649 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736a24e3-f7b7-4275-a64d-06a14782d964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:55.018537 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a24e3-f7b7-4275-a64d-06a14782d964 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:55.018830 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a24e3-f7b7-4275-a64d-06a14782d964 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:55.019146 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736a24e3-f7b7-4275-a64d-06a14782d964 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:55.031665 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c98a7db8-2fe5-4f21-807b-49cd67dba225 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:55.033061 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c98a7db8-2fe5-4f21-807b-49cd67dba225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:55.043567 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736a24e3-f7b7-4275-a64d-06a14782d964 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:55.066179 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-736a24e3-f7b7-4275-a64d-06a14782d964 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 15:00:55.066179 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:55.066179 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:55.066179 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:55.066179 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:55.066179 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:55.066179 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:55.066179 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:55.066179 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:55.066179 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:55.066179 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:55.066179 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:55.066179 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:55.066179 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:55.066179 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:55.066179 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:55.067316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:55.067316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:55.067316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:55.067316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:55.067316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:55.067316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:55.067316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:55.067316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:55.067316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:55.067316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:55.067316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:55.067316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:00:55.067316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:55.067316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:55.067316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:55.067316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:55.068181 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:55.068181 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:55.068181 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:55.068181 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:55.068181 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:55.068181 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:55.068181 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:55.068181 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:55.068181 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:55.068181 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:55.068181 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 15:00:55.068181 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:00:55.072933 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 201/405] 10.222.0.50 () {56 vars in 1114 bytes} [Fri May 27 15:00:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:00:55.077468 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736a24e3-f7b7-4275-a64d-06a14782d964 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:55.188882 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 205/406] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:00:55.214167 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a7db8-2fe5-4f21-807b-49cd67dba225 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:55.227355 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736a24e3-f7b7-4275-a64d-06a14782d964 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:55.267969 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d28809-ec0d-499d-b5ff-fd40c68c2e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:55.269094 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d28809-ec0d-499d-b5ff-fd40c68c2e03 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:55.269576 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d28809-ec0d-499d-b5ff-fd40c68c2e03 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:55.270124 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d28809-ec0d-499d-b5ff-fd40c68c2e03 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:55.271114 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d28809-ec0d-499d-b5ff-fd40c68c2e03 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:55.271625 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d28809-ec0d-499d-b5ff-fd40c68c2e03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:55.273209 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d28809-ec0d-499d-b5ff-fd40c68c2e03 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:55.282202 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:55.289484 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 202/407] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:00:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:55.294996 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d28809-ec0d-499d-b5ff-fd40c68c2e03 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:55.394167 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a7db8-2fe5-4f21-807b-49cd67dba225 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:55.460952 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e64d57a-594d-431f-aa53-eee9418e7c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:55.461808 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e64d57a-594d-431f-aa53-eee9418e7c81 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:55.462085 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e64d57a-594d-431f-aa53-eee9418e7c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:55.462350 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e64d57a-594d-431f-aa53-eee9418e7c81 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:55.468160 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:55.484968 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e64d57a-594d-431f-aa53-eee9418e7c81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:55.489194 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d28809-ec0d-499d-b5ff-fd40c68c2e03 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:55.493626 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-7e64d57a-594d-431f-aa53-eee9418e7c81 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:00:55.493626 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:55.493626 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:55.493626 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:55.493626 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:55.493626 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:55.493626 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:55.493626 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:55.493626 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:55.493626 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:55.493626 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:55.493626 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:55.493626 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:55.493626 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:55.493626 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:55.493626 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:55.494596 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:55.494596 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:55.494596 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:55.494596 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:55.494596 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:55.494596 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:55.494596 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:55.494596 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:55.494596 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:55.494596 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:55.494596 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:55.494596 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:00:55.494596 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:55.494596 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:55.494596 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:55.494596 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:55.495499 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:55.495499 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:55.495499 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:55.495499 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:55.495499 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:55.495499 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:55.495499 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:55.495499 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:55.495499 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:55.495499 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:55.495499 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:00:55.495499 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:00:55.495499 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 206/408] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 15:00:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:00:55.501500 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e64d57a-594d-431f-aa53-eee9418e7c81 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:55.568211 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf8cf0-c5f0-4df2-a38a-53a7fe6aed74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:55.569082 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf8cf0-c5f0-4df2-a38a-53a7fe6aed74 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:55.569366 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf8cf0-c5f0-4df2-a38a-53a7fe6aed74 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:55.569646 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf8cf0-c5f0-4df2-a38a-53a7fe6aed74 None admin] PATH_INFO: `/v3/groups/d49ca26b382043a0a226f9249c3326c8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:55.572056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaf8cf0-c5f0-4df2-a38a-53a7fe6aed74 None admin] RBAC: Authorizing `identity:get_group(group_id=d49ca26b382043a0a226f9249c3326c8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:55.573512 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaf8cf0-c5f0-4df2-a38a-53a7fe6aed74 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:55.575205 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 203/409] 10.222.0.50 () {56 vars in 1180 bytes} [Fri May 27 15:00:55 2022] GET /identity/v3/groups/d49ca26b382043a0a226f9249c3326c8 => generated 226 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:00:55.581559 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf8cf0-c5f0-4df2-a38a-53a7fe6aed74 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:55.687233 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:55.713027 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e64d57a-594d-431f-aa53-eee9418e7c81 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:55.724152 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf8cf0-c5f0-4df2-a38a-53a7fe6aed74 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:55.761281 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11736b0-b128-47ae-945e-ea405a6f2077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:55.762510 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11736b0-b128-47ae-945e-ea405a6f2077 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:55.763190 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11736b0-b128-47ae-945e-ea405a6f2077 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:55.763665 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11736b0-b128-47ae-945e-ea405a6f2077 None admin] PATH_INFO: `/v3/projects/65801518f2424895ab7af78ae7acb341` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:55.765197 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11736b0-b128-47ae-945e-ea405a6f2077 None admin] RBAC: Authorizing `identity:get_project(project_id=65801518f2424895ab7af78ae7acb341)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:55.767455 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11736b0-b128-47ae-945e-ea405a6f2077 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:55.769969 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 204/410] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:00:55 2022] GET /identity/v3/projects/65801518f2424895ab7af78ae7acb341 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:55.775276 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11736b0-b128-47ae-945e-ea405a6f2077 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:55.810553 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee7719b-3b52-41cf-a0a4-f72910227512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:55.811269 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7719b-3b52-41cf-a0a4-f72910227512 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:55.811561 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7719b-3b52-41cf-a0a4-f72910227512 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:55.811803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee7719b-3b52-41cf-a0a4-f72910227512 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:55.812650 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee7719b-3b52-41cf-a0a4-f72910227512 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:55.813170 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee7719b-3b52-41cf-a0a4-f72910227512 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:55.814774 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee7719b-3b52-41cf-a0a4-f72910227512 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:55.828906 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 207/411] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:00:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:55.830640 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6af5ad-cb5a-4f43-91d7-e2c495b86c9a None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:55.830968 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6af5ad-cb5a-4f43-91d7-e2c495b86c9a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:55.831244 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6af5ad-cb5a-4f43-91d7-e2c495b86c9a None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:55.832613 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 208/412] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:55.833332 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6af5ad-cb5a-4f43-91d7-e2c495b86c9a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:55.895649 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11736b0-b128-47ae-945e-ea405a6f2077 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:55.908859 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:55.936396 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca4a5cb-9f0d-4b20-879b-f3fe9fb9c936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:55.937360 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca4a5cb-9f0d-4b20-879b-f3fe9fb9c936 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:55.937714 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca4a5cb-9f0d-4b20-879b-f3fe9fb9c936 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:55.938361 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca4a5cb-9f0d-4b20-879b-f3fe9fb9c936 None admin] PATH_INFO: `/v3/projects/65801518f2424895ab7af78ae7acb341/groups/d49ca26b382043a0a226f9249c3326c8/roles/ad9498a5e2064a468acc151fb2bce49e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:55.940763 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca4a5cb-9f0d-4b20-879b-f3fe9fb9c936 None admin] RBAC: Authorizing `identity:create_grant(project_id=65801518f2424895ab7af78ae7acb341, group_id=d49ca26b382043a0a226f9249c3326c8, role_id=ad9498a5e2064a468acc151fb2bce49e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:55.942295 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca4a5cb-9f0d-4b20-879b-f3fe9fb9c936 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:55.945544 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:55.960856 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6af5ad-cb5a-4f43-91d7-e2c495b86c9a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:55.961360 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 205/413] 10.222.0.50 () {58 vars in 1442 bytes} [Fri May 27 15:00:55 2022] PUT /identity/v3/projects/65801518f2424895ab7af78ae7acb341/groups/d49ca26b382043a0a226f9249c3326c8/roles/ad9498a5e2064a468acc151fb2bce49e => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:00:55.963040 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d0feb-f933-4b7c-b5c0-dfd49d3103f2 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:55.963521 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d0feb-f933-4b7c-b5c0-dfd49d3103f2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:55.963802 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d0feb-f933-4b7c-b5c0-dfd49d3103f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:55.984030 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2d9d0feb-f933-4b7c-b5c0-dfd49d3103f2 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:55.986241 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2d9d0feb-f933-4b7c-b5c0-dfd49d3103f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:56.002082 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d1685-a4b3-4359-890f-18eda6b55996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:56.003451 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d1685-a4b3-4359-890f-18eda6b55996 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:56.003931 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d1685-a4b3-4359-890f-18eda6b55996 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:56.004449 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d1685-a4b3-4359-890f-18eda6b55996 None admin] PATH_INFO: `/v3/groups/90f7c40ee18847979e4a1a72530ac3ec` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:56.005867 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309d1685-a4b3-4359-890f-18eda6b55996 None admin] RBAC: Authorizing `identity:get_group(group_id=90f7c40ee18847979e4a1a72530ac3ec)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:56.007611 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309d1685-a4b3-4359-890f-18eda6b55996 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:56.009841 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 209/414] 10.222.0.50 () {56 vars in 1180 bytes} [Fri May 27 15:00:55 2022] GET /identity/v3/groups/90f7c40ee18847979e4a1a72530ac3ec => generated 229 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:56.019924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d1685-a4b3-4359-890f-18eda6b55996 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:56.116010 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 206/415] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:00:56.142377 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d0feb-f933-4b7c-b5c0-dfd49d3103f2 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:56.224156 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d1685-a4b3-4359-890f-18eda6b55996 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:56.254229 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:56.266472 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d0feb-f933-4b7c-b5c0-dfd49d3103f2 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:56.267706 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd63fe8-9cdc-4bba-845a-6b61f53f3ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:56.269926 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd63fe8-9cdc-4bba-845a-6b61f53f3ec0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:56.270415 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd63fe8-9cdc-4bba-845a-6b61f53f3ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:56.270880 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd63fe8-9cdc-4bba-845a-6b61f53f3ec0 None admin] PATH_INFO: `/v3/projects/6fdd45226b7e4d17bd9b93b1a5b70196` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:56.272260 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd63fe8-9cdc-4bba-845a-6b61f53f3ec0 None admin] RBAC: Authorizing `identity:get_project(project_id=6fdd45226b7e4d17bd9b93b1a5b70196)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:56.273997 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd63fe8-9cdc-4bba-845a-6b61f53f3ec0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:56.276838 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 210/416] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:00:56 2022] GET /identity/v3/projects/6fdd45226b7e4d17bd9b93b1a5b70196 => generated 344 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:00:56.283096 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd63fe8-9cdc-4bba-845a-6b61f53f3ec0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:56.309644 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:56.317634 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ad2515-bc40-4305-bcf6-56c507ecde54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:56.319128 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad2515-bc40-4305-bcf6-56c507ecde54 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:56.319493 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad2515-bc40-4305-bcf6-56c507ecde54 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:56.319845 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ad2515-bc40-4305-bcf6-56c507ecde54 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:56.334085 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ad2515-bc40-4305-bcf6-56c507ecde54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:56.345090 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-28ad2515-bc40-4305-bcf6-56c507ecde54 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 15:00:56.345090 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:56.345090 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:56.345090 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:56.345090 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:56.345090 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:56.345090 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:56.345090 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:56.345090 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:56.345090 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:56.345090 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:56.345090 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:56.345090 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:56.345090 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:56.345090 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:56.345090 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:56.346064 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:56.346064 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:56.346064 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:56.346064 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:56.346064 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:56.346064 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:56.346064 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:56.346064 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:56.346064 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:56.346064 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:56.346064 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:56.346064 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:00:56.346064 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:56.346064 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:56.346064 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:56.346064 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:56.346900 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:56.346900 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:56.346900 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:56.346900 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:56.346900 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:56.346900 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:56.346900 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:56.346900 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:56.346900 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:56.346900 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:56.346900 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 15:00:56.346900 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:00:56.346900 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 207/417] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:00:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:00:56.352917 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ad2515-bc40-4305-bcf6-56c507ecde54 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:56.478775 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ad2515-bc40-4305-bcf6-56c507ecde54 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:56.513592 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ab7f96-fb62-4488-885e-6bf5bce3a1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:56.514286 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab7f96-fb62-4488-885e-6bf5bce3a1f2 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:56.518873 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab7f96-fb62-4488-885e-6bf5bce3a1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:56.518873 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ab7f96-fb62-4488-885e-6bf5bce3a1f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:56.518873 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ab7f96-fb62-4488-885e-6bf5bce3a1f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:56.518873 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ab7f96-fb62-4488-885e-6bf5bce3a1f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:56.518873 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ab7f96-fb62-4488-885e-6bf5bce3a1f2 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:56.529095 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 208/418] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 15:00:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:56.535193 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ab7f96-fb62-4488-885e-6bf5bce3a1f2 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:56.552921 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd63fe8-9cdc-4bba-845a-6b61f53f3ec0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:56.596638 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:56.654275 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ab7f96-fb62-4488-885e-6bf5bce3a1f2 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:56.655538 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b103fc43-0bc9-41df-974e-842b3f8e225c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:56.656482 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b103fc43-0bc9-41df-974e-842b3f8e225c None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:56.656757 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b103fc43-0bc9-41df-974e-842b3f8e225c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:56.657034 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b103fc43-0bc9-41df-974e-842b3f8e225c None admin] PATH_INFO: `/v3/projects/6fdd45226b7e4d17bd9b93b1a5b70196/groups/90f7c40ee18847979e4a1a72530ac3ec/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:56.663628 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b103fc43-0bc9-41df-974e-842b3f8e225c None admin] RBAC: Authorizing `identity:create_grant(project_id=6fdd45226b7e4d17bd9b93b1a5b70196, group_id=90f7c40ee18847979e4a1a72530ac3ec, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:56.665079 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b103fc43-0bc9-41df-974e-842b3f8e225c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:56.684501 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 211/419] 10.222.0.50 () {58 vars in 1442 bytes} [Fri May 27 15:00:56 2022] PUT /identity/v3/projects/6fdd45226b7e4d17bd9b93b1a5b70196/groups/90f7c40ee18847979e4a1a72530ac3ec/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:00:56.691466 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f72c05-d063-4db5-a912-c7a273441042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:56.692133 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f72c05-d063-4db5-a912-c7a273441042 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:56.692422 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f72c05-d063-4db5-a912-c7a273441042 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:56.692702 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f72c05-d063-4db5-a912-c7a273441042 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:56.771018 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f72c05-d063-4db5-a912-c7a273441042 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:56.772500 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f72c05-d063-4db5-a912-c7a273441042 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:56.792278 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dfbf9d-e05f-4398-9143-fe259a7fe711 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:56.792603 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dfbf9d-e05f-4398-9143-fe259a7fe711 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:56.792833 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dfbf9d-e05f-4398-9143-fe259a7fe711 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:56.794533 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 212/420] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:56.801170 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f70c6d-5275-4b41-9c97-6e5356b5aed7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:56.801454 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f70c6d-5275-4b41-9c97-6e5356b5aed7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:56.801728 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f70c6d-5275-4b41-9c97-6e5356b5aed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:56.819790 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c1f70c6d-5275-4b41-9c97-6e5356b5aed7 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:56.821234 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c1f70c6d-5275-4b41-9c97-6e5356b5aed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:56.826989 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:56.880455 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:56.888849 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 209/421] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:00:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 15:00:56.895378 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f72c05-d063-4db5-a912-c7a273441042 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:56.900691 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 213/422] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 15:00:56.927261 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f70c6d-5275-4b41-9c97-6e5356b5aed7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:57.006123 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f72c05-d063-4db5-a912-c7a273441042 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:57.043109 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4188fc-430f-4122-9ad8-1fa3b91def11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:57.044241 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4188fc-430f-4122-9ad8-1fa3b91def11 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:57.044786 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4188fc-430f-4122-9ad8-1fa3b91def11 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:57.045063 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4188fc-430f-4122-9ad8-1fa3b91def11 None admin] PATH_INFO: `/v3/projects/00203c7cc02341ee8ee7a591095f0c7d` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:57.046415 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f70c6d-5275-4b41-9c97-6e5356b5aed7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:57.047614 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4188fc-430f-4122-9ad8-1fa3b91def11 None admin] RBAC: Authorizing `identity:get_project(project_id=00203c7cc02341ee8ee7a591095f0c7d)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:57.049202 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4188fc-430f-4122-9ad8-1fa3b91def11 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:57.052116 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 210/423] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:00:56 2022] GET /identity/v3/projects/00203c7cc02341ee8ee7a591095f0c7d => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:00:57.060535 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4188fc-430f-4122-9ad8-1fa3b91def11 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:57.087576 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:57.099240 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd4ee3-7d7c-4743-8967-775a99c9b189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:57.100522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd4ee3-7d7c-4743-8967-775a99c9b189 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:57.101340 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd4ee3-7d7c-4743-8967-775a99c9b189 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:57.102092 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd4ee3-7d7c-4743-8967-775a99c9b189 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:57.116227 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dd4ee3-7d7c-4743-8967-775a99c9b189 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:57.125428 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-f7dd4ee3-7d7c-4743-8967-775a99c9b189 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 15:00:57.125428 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:57.125428 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:57.125428 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:57.125428 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:57.125428 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:57.125428 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:57.125428 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:57.125428 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:57.125428 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:57.125428 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:57.125428 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:57.125428 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:57.125428 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:57.125428 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:57.125428 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:57.126415 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:57.126415 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:57.126415 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:57.126415 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:57.126415 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:57.126415 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:57.126415 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:57.126415 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:57.126415 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:57.126415 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:57.126415 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:57.126415 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:00:57.126415 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:57.126415 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:57.126415 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:57.126415 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:57.127318 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:57.127318 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:57.127318 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:57.127318 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:57.127318 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:57.127318 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:57.127318 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:57.127318 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:57.127318 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:57.127318 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:57.127318 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 15:00:57.127318 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:00:57.127318 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 214/424] 10.222.0.50 () {56 vars in 1114 bytes} [Fri May 27 15:00:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:00:57.132585 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd4ee3-7d7c-4743-8967-775a99c9b189 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:57.187752 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:57.190570 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4188fc-430f-4122-9ad8-1fa3b91def11 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:57.230915 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14d1795-997b-42bb-8f88-2ebfebb2f5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:57.232701 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d1795-997b-42bb-8f88-2ebfebb2f5a7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:57.233223 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d1795-997b-42bb-8f88-2ebfebb2f5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:57.233974 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d1795-997b-42bb-8f88-2ebfebb2f5a7 None admin] PATH_INFO: `/v3/groups/d49ca26b382043a0a226f9249c3326c8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:57.235772 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14d1795-997b-42bb-8f88-2ebfebb2f5a7 None admin] RBAC: Authorizing `identity:get_group(group_id=d49ca26b382043a0a226f9249c3326c8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:57.238459 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14d1795-997b-42bb-8f88-2ebfebb2f5a7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:57.241593 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 211/425] 10.222.0.50 () {56 vars in 1180 bytes} [Fri May 27 15:00:57 2022] GET /identity/v3/groups/d49ca26b382043a0a226f9249c3326c8 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:57.245897 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd4ee3-7d7c-4743-8967-775a99c9b189 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:57.248046 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14d1795-997b-42bb-8f88-2ebfebb2f5a7 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:57.281679 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf8a25b-e5cd-4652-bfde-2d882364c14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:57.282390 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf8a25b-e5cd-4652-bfde-2d882364c14a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:57.282671 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf8a25b-e5cd-4652-bfde-2d882364c14a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:57.282971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf8a25b-e5cd-4652-bfde-2d882364c14a None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:57.283709 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf8a25b-e5cd-4652-bfde-2d882364c14a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:57.284012 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf8a25b-e5cd-4652-bfde-2d882364c14a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:57.285503 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf8a25b-e5cd-4652-bfde-2d882364c14a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:57.298829 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 215/426] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:00:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:00:57.300155 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf8a25b-e5cd-4652-bfde-2d882364c14a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:57.431452 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:57.439788 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf8a25b-e5cd-4652-bfde-2d882364c14a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:57.451133 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14d1795-997b-42bb-8f88-2ebfebb2f5a7 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:57.454449 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:57.496641 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af5c4f7-18a6-4377-b2a2-ec97284d3ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:57.497306 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af5c4f7-18a6-4377-b2a2-ec97284d3ee1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:57.497690 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af5c4f7-18a6-4377-b2a2-ec97284d3ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:57.498124 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af5c4f7-18a6-4377-b2a2-ec97284d3ee1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:57.499025 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af5c4f7-18a6-4377-b2a2-ec97284d3ee1 None admin] RBAC: Adding query filter params (group.id=d49ca26b382043a0a226f9249c3326c8, role.id=8442c42e847a4b38b76f3a7fbe9ada85, scope.project.id=00203c7cc02341ee8ee7a591095f0c7d) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:57.499707 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af5c4f7-18a6-4377-b2a2-ec97284d3ee1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:57.502056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af5c4f7-18a6-4377-b2a2-ec97284d3ee1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:57.509574 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 212/427] 10.222.0.50 () {56 vars in 1376 bytes} [Fri May 27 15:00:57 2022] GET /identity/v3/role_assignments?group.id=d49ca26b382043a0a226f9249c3326c8&scope.project.id=00203c7cc02341ee8ee7a591095f0c7d&role.id=8442c42e847a4b38b76f3a7fbe9ada85 => generated 611 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:00:57.528687 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcab308-77e3-4802-987b-c631f5a8b8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:57.529360 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcab308-77e3-4802-987b-c631f5a8b8f5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:57.529641 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcab308-77e3-4802-987b-c631f5a8b8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:57.529914 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcab308-77e3-4802-987b-c631f5a8b8f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:57.632889 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcab308-77e3-4802-987b-c631f5a8b8f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:57.634302 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcab308-77e3-4802-987b-c631f5a8b8f5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:57.700074 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed105d1f-c667-4c4b-8543-f6ef4758760d None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:57.700381 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed105d1f-c667-4c4b-8543-f6ef4758760d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:57.700640 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed105d1f-c667-4c4b-8543-f6ef4758760d None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:57.702651 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 213/428] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:00:57.709613 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895fd964-6f37-4922-a6ec-8961c81f3bee None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:57.709902 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895fd964-6f37-4922-a6ec-8961c81f3bee None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:57.710182 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895fd964-6f37-4922-a6ec-8961c81f3bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:57.735198 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-895fd964-6f37-4922-a6ec-8961c81f3bee None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:57.736986 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-895fd964-6f37-4922-a6ec-8961c81f3bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:57.791245 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 216/429] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:00:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 15:00:57.796967 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcab308-77e3-4802-987b-c631f5a8b8f5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:57.808358 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:57.858336 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 214/430] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:00:57.883904 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895fd964-6f37-4922-a6ec-8961c81f3bee None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:57.958802 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcab308-77e3-4802-987b-c631f5a8b8f5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:58.004211 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:58.007205 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eef412f-0f39-4d79-a180-dcdbaad50ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:58.008675 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef412f-0f39-4d79-a180-dcdbaad50ebd None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:58.009473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef412f-0f39-4d79-a180-dcdbaad50ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:58.009945 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef412f-0f39-4d79-a180-dcdbaad50ebd None admin] PATH_INFO: `/v3/projects/00203c7cc02341ee8ee7a591095f0c7d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:58.011540 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eef412f-0f39-4d79-a180-dcdbaad50ebd None admin] RBAC: Authorizing `identity:get_project(project_id=00203c7cc02341ee8ee7a591095f0c7d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:58.014318 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eef412f-0f39-4d79-a180-dcdbaad50ebd None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:58.017377 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 217/431] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:00:57 2022] GET /identity/v3/projects/00203c7cc02341ee8ee7a591095f0c7d => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:58.022420 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eef412f-0f39-4d79-a180-dcdbaad50ebd None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:58.040798 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895fd964-6f37-4922-a6ec-8961c81f3bee None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:58.090088 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c37145f-6cf3-444b-b4c0-f0855631f4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:58.091397 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37145f-6cf3-444b-b4c0-f0855631f4d9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:58.091892 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37145f-6cf3-444b-b4c0-f0855631f4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:58.092415 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c37145f-6cf3-444b-b4c0-f0855631f4d9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:58.107378 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c37145f-6cf3-444b-b4c0-f0855631f4d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:58.118674 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-9c37145f-6cf3-444b-b4c0-f0855631f4d9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 27 15:00:58.118674 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:58.118674 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:58.118674 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:58.118674 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:58.118674 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:58.118674 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:58.118674 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:58.118674 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:58.118674 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:58.118674 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:58.118674 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:58.118674 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:58.118674 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:58.118674 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:58.118674 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:58.119628 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:58.119628 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:58.119628 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:58.119628 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:58.119628 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:58.119628 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:58.119628 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:58.119628 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:58.119628 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:58.119628 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:58.119628 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:58.119628 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:00:58.119628 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:58.119628 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:58.119628 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:58.119628 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:58.120443 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:58.120443 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:58.120443 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:58.120443 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:58.120443 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:58.120443 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:58.120443 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:58.120443 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:58.120443 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:58.120443 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:58.120443 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 27 15:00:58.120443 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:00:58.123081 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 215/432] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:00:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:00:58.129596 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c37145f-6cf3-444b-b4c0-f0855631f4d9 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:58.150729 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:58.196678 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eef412f-0f39-4d79-a180-dcdbaad50ebd None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:58.255138 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff25f658-0abb-4abd-91c2-8a1d1f953011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:58.256055 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff25f658-0abb-4abd-91c2-8a1d1f953011 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:58.256466 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff25f658-0abb-4abd-91c2-8a1d1f953011 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:58.256747 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff25f658-0abb-4abd-91c2-8a1d1f953011 None admin] PATH_INFO: `/v3/groups/d49ca26b382043a0a226f9249c3326c8` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:58.258029 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff25f658-0abb-4abd-91c2-8a1d1f953011 None admin] RBAC: Authorizing `identity:get_group(group_id=d49ca26b382043a0a226f9249c3326c8)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:58.260020 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff25f658-0abb-4abd-91c2-8a1d1f953011 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:58.262295 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 218/433] 10.222.0.50 () {56 vars in 1180 bytes} [Fri May 27 15:00:58 2022] GET /identity/v3/groups/d49ca26b382043a0a226f9249c3326c8 => generated 226 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:58.268376 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff25f658-0abb-4abd-91c2-8a1d1f953011 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:58.332239 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c37145f-6cf3-444b-b4c0-f0855631f4d9 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:58.379783 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0e2830-d6b4-4e5e-9243-8f21584747d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:58.380569 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e2830-d6b4-4e5e-9243-8f21584747d6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:58.380852 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e2830-d6b4-4e5e-9243-8f21584747d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:58.381123 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e2830-d6b4-4e5e-9243-8f21584747d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:58.381887 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0e2830-d6b4-4e5e-9243-8f21584747d6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:58.382282 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0e2830-d6b4-4e5e-9243-8f21584747d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:58.383991 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0e2830-d6b4-4e5e-9243-8f21584747d6 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:58.396658 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:58.403721 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 216/434] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 15:00:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:58.409690 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0e2830-d6b4-4e5e-9243-8f21584747d6 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:58.435366 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff25f658-0abb-4abd-91c2-8a1d1f953011 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:58.478074 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef63348c-3584-4b6e-b4c9-586fe5c277d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:58.478714 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63348c-3584-4b6e-b4c9-586fe5c277d7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:58.479016 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63348c-3584-4b6e-b4c9-586fe5c277d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:58.479283 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef63348c-3584-4b6e-b4c9-586fe5c277d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:58.480764 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef63348c-3584-4b6e-b4c9-586fe5c277d7 None admin] RBAC: Adding query filter params (group.id=d49ca26b382043a0a226f9249c3326c8, role.id=ad9498a5e2064a468acc151fb2bce49e, scope.project.id=00203c7cc02341ee8ee7a591095f0c7d) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:58.480764 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef63348c-3584-4b6e-b4c9-586fe5c277d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:58.481805 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef63348c-3584-4b6e-b4c9-586fe5c277d7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:58.495179 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 219/435] 10.222.0.50 () {56 vars in 1376 bytes} [Fri May 27 15:00:58 2022] GET /identity/v3/role_assignments?group.id=d49ca26b382043a0a226f9249c3326c8&scope.project.id=00203c7cc02341ee8ee7a591095f0c7d&role.id=ad9498a5e2064a468acc151fb2bce49e => generated 611 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:00:58.587488 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0e2830-d6b4-4e5e-9243-8f21584747d6 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:58.641814 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4745650c-cfd5-4005-8ad7-1af2c58ec039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:58.643166 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4745650c-cfd5-4005-8ad7-1af2c58ec039 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:58.643700 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4745650c-cfd5-4005-8ad7-1af2c58ec039 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:58.644184 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4745650c-cfd5-4005-8ad7-1af2c58ec039 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:58.743289 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:58.746761 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4745650c-cfd5-4005-8ad7-1af2c58ec039 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:58.749272 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4745650c-cfd5-4005-8ad7-1af2c58ec039 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:58.838990 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a6e7b0-a24e-4bca-b95b-ecee890ebac4 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:58.839791 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a6e7b0-a24e-4bca-b95b-ecee890ebac4 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:58.840025 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a6e7b0-a24e-4bca-b95b-ecee890ebac4 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:58.843455 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 220/436] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:58 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 15:00:58.850271 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7e9e5-3f14-4bc4-bc70-57bd1f16bb3d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:58.850570 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7e9e5-3f14-4bc4-bc70-57bd1f16bb3d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:58.850895 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b7e9e5-3f14-4bc4-bc70-57bd1f16bb3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:58.866242 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:58.875647 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-36b7e9e5-3f14-4bc4-bc70-57bd1f16bb3d None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:58.877928 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-36b7e9e5-3f14-4bc4-bc70-57bd1f16bb3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:58.914351 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 217/437] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:00:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 15:00:58.919853 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4745650c-cfd5-4005-8ad7-1af2c58ec039 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:58.981206 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 221/438] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:00:59.004612 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b7e9e5-3f14-4bc4-bc70-57bd1f16bb3d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:59.042636 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:59.045699 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4745650c-cfd5-4005-8ad7-1af2c58ec039 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:59.086864 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e8789-dfd9-4e4c-a89e-fae8c9f00569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:59.087727 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e8789-dfd9-4e4c-a89e-fae8c9f00569 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:59.088282 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e8789-dfd9-4e4c-a89e-fae8c9f00569 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:59.088541 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e8789-dfd9-4e4c-a89e-fae8c9f00569 None admin] PATH_INFO: `/v3/projects/65801518f2424895ab7af78ae7acb341` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:59.089845 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14e8789-dfd9-4e4c-a89e-fae8c9f00569 None admin] RBAC: Authorizing `identity:get_project(project_id=65801518f2424895ab7af78ae7acb341)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:59.091457 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14e8789-dfd9-4e4c-a89e-fae8c9f00569 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:59.098206 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 218/439] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:00:58 2022] GET /identity/v3/projects/65801518f2424895ab7af78ae7acb341 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:00:59.099148 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e8789-dfd9-4e4c-a89e-fae8c9f00569 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:59.114236 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b7e9e5-3f14-4bc4-bc70-57bd1f16bb3d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:59.151933 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec8480-76cb-4153-93d5-75530c0fbedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:59.152702 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec8480-76cb-4153-93d5-75530c0fbedb None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:59.152988 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec8480-76cb-4153-93d5-75530c0fbedb None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:59.153264 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec8480-76cb-4153-93d5-75530c0fbedb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:59.157994 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:59.166821 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ec8480-76cb-4153-93d5-75530c0fbedb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:59.174949 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-d9ec8480-76cb-4153-93d5-75530c0fbedb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 15:00:59.174949 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:59.174949 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:59.174949 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:59.174949 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:59.174949 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:59.174949 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:59.174949 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:59.174949 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:59.174949 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:59.174949 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:59.174949 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:59.174949 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:59.174949 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:59.174949 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:59.174949 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:59.176084 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:59.176084 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:59.176084 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:59.176084 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:59.176084 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:59.176084 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:59.176084 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:59.176084 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:59.176084 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:59.176084 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:59.176084 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:59.176084 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:00:59.176084 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:59.176084 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:59.176084 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:59.176084 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:59.176941 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:59.176941 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:59.176941 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:59.176941 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:59.176941 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:59.176941 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:59.176941 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:59.176941 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:59.176941 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:59.176941 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:59.176941 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 15:00:59.176941 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:00:59.176941 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 222/440] 10.222.0.50 () {56 vars in 1114 bytes} [Fri May 27 15:00:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:00:59.181683 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec8480-76cb-4153-93d5-75530c0fbedb None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:59.219223 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e8789-dfd9-4e4c-a89e-fae8c9f00569 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:59.257516 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b619a503-b146-4ba0-a471-2d2921ec3c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:59.258306 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619a503-b146-4ba0-a471-2d2921ec3c44 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:59.258592 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619a503-b146-4ba0-a471-2d2921ec3c44 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:59.258892 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619a503-b146-4ba0-a471-2d2921ec3c44 None admin] PATH_INFO: `/v3/groups/d49ca26b382043a0a226f9249c3326c8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:59.260273 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b619a503-b146-4ba0-a471-2d2921ec3c44 None admin] RBAC: Authorizing `identity:get_group(group_id=d49ca26b382043a0a226f9249c3326c8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:59.261756 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b619a503-b146-4ba0-a471-2d2921ec3c44 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:59.263686 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 219/441] 10.222.0.50 () {56 vars in 1180 bytes} [Fri May 27 15:00:59 2022] GET /identity/v3/groups/d49ca26b382043a0a226f9249c3326c8 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:00:59.270085 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b619a503-b146-4ba0-a471-2d2921ec3c44 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:59.289536 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec8480-76cb-4153-93d5-75530c0fbedb None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:59.306707 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:59.325048 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1c960-b099-40c3-971b-8af8c4ca17e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:59.325720 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1c960-b099-40c3-971b-8af8c4ca17e9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:59.326002 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1c960-b099-40c3-971b-8af8c4ca17e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:59.326279 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1c960-b099-40c3-971b-8af8c4ca17e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:59.327065 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a1c960-b099-40c3-971b-8af8c4ca17e9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:59.327370 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a1c960-b099-40c3-971b-8af8c4ca17e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:59.329559 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a1c960-b099-40c3-971b-8af8c4ca17e9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:59.338893 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 223/442] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:00:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:00:59.344479 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1c960-b099-40c3-971b-8af8c4ca17e9 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:59.376869 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b619a503-b146-4ba0-a471-2d2921ec3c44 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:59.404951 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:59.414679 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca64b0bd-4f47-4f5f-98c0-5f2c90d289d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:59.415369 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca64b0bd-4f47-4f5f-98c0-5f2c90d289d4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:59.415654 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca64b0bd-4f47-4f5f-98c0-5f2c90d289d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:59.415931 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca64b0bd-4f47-4f5f-98c0-5f2c90d289d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:59.416751 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca64b0bd-4f47-4f5f-98c0-5f2c90d289d4 None admin] RBAC: Adding query filter params (group.id=d49ca26b382043a0a226f9249c3326c8, role.id=8442c42e847a4b38b76f3a7fbe9ada85, scope.project.id=65801518f2424895ab7af78ae7acb341) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:59.417063 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca64b0bd-4f47-4f5f-98c0-5f2c90d289d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:59.418485 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca64b0bd-4f47-4f5f-98c0-5f2c90d289d4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:59.426889 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 220/443] 10.222.0.50 () {56 vars in 1376 bytes} [Fri May 27 15:00:59 2022] GET /identity/v3/role_assignments?group.id=d49ca26b382043a0a226f9249c3326c8&scope.project.id=65801518f2424895ab7af78ae7acb341&role.id=8442c42e847a4b38b76f3a7fbe9ada85 => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:00:59.434150 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456bca9-3d17-41e2-9afd-9c03c1f87ce1 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:59.434444 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456bca9-3d17-41e2-9afd-9c03c1f87ce1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:59.434728 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456bca9-3d17-41e2-9afd-9c03c1f87ce1 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:59.436211 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 221/444] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:00:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 15:00:59.442978 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a085c7de-9f77-4d99-a784-c3211ff08da2 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:59.443262 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a085c7de-9f77-4d99-a784-c3211ff08da2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:59.443545 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a085c7de-9f77-4d99-a784-c3211ff08da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:59.456562 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1c960-b099-40c3-971b-8af8c4ca17e9 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:59.461318 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a085c7de-9f77-4d99-a784-c3211ff08da2 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:00:59.462534 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a085c7de-9f77-4d99-a784-c3211ff08da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:00:59.497239 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cae533-9ea0-4c32-b031-87e0081d747b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:59.498176 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cae533-9ea0-4c32-b031-87e0081d747b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:59.498459 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cae533-9ea0-4c32-b031-87e0081d747b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:59.498739 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cae533-9ea0-4c32-b031-87e0081d747b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:59.541280 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 222/445] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:00:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 15:00:59.565717 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a085c7de-9f77-4d99-a784-c3211ff08da2 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:59.580883 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cae533-9ea0-4c32-b031-87e0081d747b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:59.582356 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cae533-9ea0-4c32-b031-87e0081d747b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:59.599713 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:59.650270 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:59.691868 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a085c7de-9f77-4d99-a784-c3211ff08da2 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:59.707591 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 224/446] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:00:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 15:00:59.713626 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cae533-9ea0-4c32-b031-87e0081d747b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:59.731067 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817d842b-83c3-4736-87ea-63bbea3d9fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:59.731956 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d842b-83c3-4736-87ea-63bbea3d9fdc None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:59.732311 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d842b-83c3-4736-87ea-63bbea3d9fdc None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:59.732739 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d842b-83c3-4736-87ea-63bbea3d9fdc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:59.746991 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817d842b-83c3-4736-87ea-63bbea3d9fdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:00:59.756382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-817d842b-83c3-4736-87ea-63bbea3d9fdc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:00:59.756382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:00:59.756382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:00:59.756382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:00:59.756382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:00:59.756382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:00:59.756382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:00:59.756382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:00:59.756382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:00:59.756382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:00:59.756382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:00:59.756382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:00:59.756382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:00:59.756382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:00:59.756382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:00:59.756382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:00:59.757332 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:00:59.757332 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:00:59.757332 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:00:59.757332 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:00:59.757332 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:00:59.757332 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:00:59.757332 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:00:59.757332 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:00:59.757332 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:00:59.757332 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:00:59.757332 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:00:59.757332 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:00:59.757332 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:00:59.757332 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:00:59.757332 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:00:59.757332 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:00:59.758277 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:00:59.758277 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:00:59.758277 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:00:59.758277 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:00:59.758277 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:00:59.758277 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:00:59.758277 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:00:59.758277 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:00:59.758277 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:00:59.758277 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:00:59.758277 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:00:59.758277 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:00:59.758277 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 223/447] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 15:00:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:00:59.764271 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817d842b-83c3-4736-87ea-63bbea3d9fdc None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:59.831995 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cae533-9ea0-4c32-b031-87e0081d747b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:59.867059 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cc35c4-b8e1-4fc3-b8b2-0d94c03467d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:59.867922 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc35c4-b8e1-4fc3-b8b2-0d94c03467d8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:59.868199 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc35c4-b8e1-4fc3-b8b2-0d94c03467d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:59.868473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc35c4-b8e1-4fc3-b8b2-0d94c03467d8 None admin] PATH_INFO: `/v3/projects/65801518f2424895ab7af78ae7acb341` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:59.868866 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:59.869696 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cc35c4-b8e1-4fc3-b8b2-0d94c03467d8 None admin] RBAC: Authorizing `identity:get_project(project_id=65801518f2424895ab7af78ae7acb341)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:59.871298 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cc35c4-b8e1-4fc3-b8b2-0d94c03467d8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:59.873042 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 225/448] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:00:59 2022] GET /identity/v3/projects/65801518f2424895ab7af78ae7acb341 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:00:59.878550 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cc35c4-b8e1-4fc3-b8b2-0d94c03467d8 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:59.879371 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817d842b-83c3-4736-87ea-63bbea3d9fdc None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:00:59.905023 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:00:59.913928 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621d9aff-3fc1-44cd-ac33-b6b9814ac910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:00:59.914600 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d9aff-3fc1-44cd-ac33-b6b9814ac910 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:00:59.914915 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d9aff-3fc1-44cd-ac33-b6b9814ac910 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:00:59.915194 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d9aff-3fc1-44cd-ac33-b6b9814ac910 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:00:59.915940 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621d9aff-3fc1-44cd-ac33-b6b9814ac910 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:00:59.916249 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621d9aff-3fc1-44cd-ac33-b6b9814ac910 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:00:59.917630 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621d9aff-3fc1-44cd-ac33-b6b9814ac910 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:00:59.926419 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 224/449] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:00:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:00:59.931847 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621d9aff-3fc1-44cd-ac33-b6b9814ac910 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:00:59.979424 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cc35c4-b8e1-4fc3-b8b2-0d94c03467d8 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:00.012529 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eef9d0-1c05-4997-86fe-6dafddaecfcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:00.013575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eef9d0-1c05-4997-86fe-6dafddaecfcc None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:00.013855 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eef9d0-1c05-4997-86fe-6dafddaecfcc None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:00.014132 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eef9d0-1c05-4997-86fe-6dafddaecfcc None admin] PATH_INFO: `/v3/groups/d49ca26b382043a0a226f9249c3326c8` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:00.015329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eef9d0-1c05-4997-86fe-6dafddaecfcc None admin] RBAC: Authorizing `identity:get_group(group_id=d49ca26b382043a0a226f9249c3326c8)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:00.016898 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eef9d0-1c05-4997-86fe-6dafddaecfcc None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:00.018521 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 226/450] 10.222.0.50 () {56 vars in 1180 bytes} [Fri May 27 15:00:59 2022] GET /identity/v3/groups/d49ca26b382043a0a226f9249c3326c8 => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:01:00.024311 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eef9d0-1c05-4997-86fe-6dafddaecfcc None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:00.030965 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621d9aff-3fc1-44cd-ac33-b6b9814ac910 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:00.065766 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad7e24b-de4e-4f78-aa8a-f8a467dc4df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:00.066410 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad7e24b-de4e-4f78-aa8a-f8a467dc4df4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:00.066687 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad7e24b-de4e-4f78-aa8a-f8a467dc4df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:00.066997 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad7e24b-de4e-4f78-aa8a-f8a467dc4df4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:00.088060 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:00.118458 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:00.125751 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eef9d0-1c05-4997-86fe-6dafddaecfcc None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:00.137540 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad7e24b-de4e-4f78-aa8a-f8a467dc4df4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:00.139033 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad7e24b-de4e-4f78-aa8a-f8a467dc4df4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:00.160187 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b40bda-451a-44d3-8075-da62b6b2d629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:00.160841 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b40bda-451a-44d3-8075-da62b6b2d629 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:00.161126 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b40bda-451a-44d3-8075-da62b6b2d629 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:00.161403 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b40bda-451a-44d3-8075-da62b6b2d629 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:00.162228 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b40bda-451a-44d3-8075-da62b6b2d629 None admin] RBAC: Adding query filter params (group.id=d49ca26b382043a0a226f9249c3326c8, role.id=ad9498a5e2064a468acc151fb2bce49e, scope.project.id=65801518f2424895ab7af78ae7acb341) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:00.162540 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b40bda-451a-44d3-8075-da62b6b2d629 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:00.164013 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b40bda-451a-44d3-8075-da62b6b2d629 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:00.169811 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 227/451] 10.222.0.50 () {56 vars in 1376 bytes} [Fri May 27 15:01:00 2022] GET /identity/v3/role_assignments?group.id=d49ca26b382043a0a226f9249c3326c8&scope.project.id=65801518f2424895ab7af78ae7acb341&role.id=ad9498a5e2064a468acc151fb2bce49e => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:01:00.247434 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 225/452] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:00:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 27 15:01:00.253291 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b40bda-451a-44d3-8075-da62b6b2d629 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:00.356719 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b40bda-451a-44d3-8075-da62b6b2d629 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:00.389609 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfb74af-f096-47bd-a27c-ec7c23d650d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:00.391103 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb74af-f096-47bd-a27c-ec7c23d650d2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:00.391396 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb74af-f096-47bd-a27c-ec7c23d650d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:00.391670 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfb74af-f096-47bd-a27c-ec7c23d650d2 None admin] PATH_INFO: `/v3/projects/6fdd45226b7e4d17bd9b93b1a5b70196` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:00.392844 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfb74af-f096-47bd-a27c-ec7c23d650d2 None admin] RBAC: Authorizing `identity:get_project(project_id=6fdd45226b7e4d17bd9b93b1a5b70196)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:00.394268 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfb74af-f096-47bd-a27c-ec7c23d650d2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:00.395963 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 228/453] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:01:00 2022] GET /identity/v3/projects/6fdd45226b7e4d17bd9b93b1a5b70196 => generated 344 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:00.401611 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad7e24b-de4e-4f78-aa8a-f8a467dc4df4 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:00.456133 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:00.508703 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad7e24b-de4e-4f78-aa8a-f8a467dc4df4 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:00.544507 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec9960-0540-4801-b19f-554788d43d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:00.545366 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec9960-0540-4801-b19f-554788d43d3d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:00.545645 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec9960-0540-4801-b19f-554788d43d3d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:00.545916 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec9960-0540-4801-b19f-554788d43d3d None admin] PATH_INFO: `/v3/groups/90f7c40ee18847979e4a1a72530ac3ec` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:00.547104 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ec9960-0540-4801-b19f-554788d43d3d None admin] RBAC: Authorizing `identity:get_group(group_id=90f7c40ee18847979e4a1a72530ac3ec)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:00.548604 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ec9960-0540-4801-b19f-554788d43d3d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:00.550139 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 226/454] 10.222.0.50 () {56 vars in 1180 bytes} [Fri May 27 15:01:00 2022] GET /identity/v3/groups/90f7c40ee18847979e4a1a72530ac3ec => generated 229 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:00.555903 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfb74af-f096-47bd-a27c-ec7c23d650d2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:00.583512 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:00.652147 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfb74af-f096-47bd-a27c-ec7c23d650d2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:00.683483 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5dd2b4-b8f1-4af6-9031-d572246b3679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:00.684132 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5dd2b4-b8f1-4af6-9031-d572246b3679 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:00.684416 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5dd2b4-b8f1-4af6-9031-d572246b3679 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:00.684694 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5dd2b4-b8f1-4af6-9031-d572246b3679 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:00.685515 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5dd2b4-b8f1-4af6-9031-d572246b3679 None admin] RBAC: Adding query filter params (group.id=90f7c40ee18847979e4a1a72530ac3ec, role.id=759d845eb6284e42931b4265af448ebc, scope.project.id=6fdd45226b7e4d17bd9b93b1a5b70196) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:00.685826 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5dd2b4-b8f1-4af6-9031-d572246b3679 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:00.687777 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5dd2b4-b8f1-4af6-9031-d572246b3679 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:00.693571 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 229/455] 10.222.0.50 () {56 vars in 1376 bytes} [Fri May 27 15:01:00 2022] GET /identity/v3/role_assignments?group.id=90f7c40ee18847979e4a1a72530ac3ec&scope.project.id=6fdd45226b7e4d17bd9b93b1a5b70196&role.id=759d845eb6284e42931b4265af448ebc => generated 611 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:03.346494 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f45a2a-5f64-4cb2-abfd-d97efee7bc9b None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:03.347089 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f45a2a-5f64-4cb2-abfd-d97efee7bc9b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:03.347089 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f45a2a-5f64-4cb2-abfd-d97efee7bc9b None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:03.348901 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 227/456] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:03.354811 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b2ccf0-4603-4bad-bf69-b2e482351762 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:03.355349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b2ccf0-4603-4bad-bf69-b2e482351762 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:03.355453 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b2ccf0-4603-4bad-bf69-b2e482351762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:03.383934 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-10b2ccf0-4603-4bad-bf69-b2e482351762 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:03.385130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-10b2ccf0-4603-4bad-bf69-b2e482351762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:03.494672 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 230/457] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:01:03.538500 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f45a2a-5f64-4cb2-abfd-d97efee7bc9b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:03.561465 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7d06fb-b16d-4b72-80ec-ae0799b65684 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:03.561873 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7d06fb-b16d-4b72-80ec-ae0799b65684 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:03.562194 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7d06fb-b16d-4b72-80ec-ae0799b65684 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:03.563628 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 231/458] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:03.568908 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f966ec-fc45-4f2d-9cc8-683f6544eeb2 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:03.569196 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f966ec-fc45-4f2d-9cc8-683f6544eeb2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:03.569474 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f966ec-fc45-4f2d-9cc8-683f6544eeb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:03.599873 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-47f966ec-fc45-4f2d-9cc8-683f6544eeb2 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:03.601069 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-47f966ec-fc45-4f2d-9cc8-683f6544eeb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:03.631384 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:03.660440 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f45a2a-5f64-4cb2-abfd-d97efee7bc9b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:03.683456 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:03.699152 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e4a737-be38-4947-a000-6133bb8e2a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:03.700029 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4a737-be38-4947-a000-6133bb8e2a03 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:03.700383 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4a737-be38-4947-a000-6133bb8e2a03 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:03.700728 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4a737-be38-4947-a000-6133bb8e2a03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:03.704803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 232/459] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 15:01:03.706164 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d6fd8-d5bf-48bd-a962-b2c3057fb9bf None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:03.706452 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d6fd8-d5bf-48bd-a962-b2c3057fb9bf None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:03.706731 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d6fd8-d5bf-48bd-a962-b2c3057fb9bf None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:03.710923 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 233/460] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:03.714368 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd420a7-0da1-40c4-8997-fd9725c3c769 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:03.714653 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd420a7-0da1-40c4-8997-fd9725c3c769 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:03.716884 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd420a7-0da1-40c4-8997-fd9725c3c769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:03.728048 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e4a737-be38-4947-a000-6133bb8e2a03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:03.729521 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e4a737-be38-4947-a000-6133bb8e2a03 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:03.742059 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9fd420a7-0da1-40c4-8997-fd9725c3c769 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:03.744575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9fd420a7-0da1-40c4-8997-fd9725c3c769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:03.754767 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-10e4a737-be38-4947-a000-6133bb8e2a03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:03.754767 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:03.754767 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:03.754767 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:03.754767 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:03.754767 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:03.754767 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:03.754767 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:03.754767 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:03.754767 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:03.754767 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:03.754767 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:03.754767 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:03.754767 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:03.754767 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:03.755782 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:03.755782 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:03.755782 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:03.755782 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:03.755782 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:03.755782 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:03.755782 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:03.755782 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:03.755782 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:03.755782 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:03.755782 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:03.755782 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:03.755782 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:03.755782 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:03.755782 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:03.755782 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:03.755782 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:03.756645 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:03.756645 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:03.756645 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:03.756645 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:03.756645 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 228/461] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:03.761951 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e4a737-be38-4947-a000-6133bb8e2a03 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:03.829166 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 234/462] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 15:01:03.830108 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd420a7-0da1-40c4-8997-fd9725c3c769 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:03.896158 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e4a737-be38-4947-a000-6133bb8e2a03 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:03.925299 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:03.934768 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d15962-145b-46c2-ac52-8a7c3419900e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:03.935882 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d15962-145b-46c2-ac52-8a7c3419900e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:03.936373 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d15962-145b-46c2-ac52-8a7c3419900e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:03.936881 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d15962-145b-46c2-ac52-8a7c3419900e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:03.946324 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd420a7-0da1-40c4-8997-fd9725c3c769 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:03.949299 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d15962-145b-46c2-ac52-8a7c3419900e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:03.951055 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d15962-145b-46c2-ac52-8a7c3419900e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:03.970907 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-d2d15962-145b-46c2-ac52-8a7c3419900e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:03.970907 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:03.970907 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:03.970907 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:03.970907 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:03.970907 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:03.970907 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:03.970907 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:03.970907 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:03.970907 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:03.970907 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:03.970907 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:03.970907 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:03.970907 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:03.970907 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:03.971804 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:03.971804 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:03.971804 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:03.971804 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:03.971804 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:03.971804 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:03.971804 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:03.971804 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:03.971804 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:03.971804 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:03.971804 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:03.971804 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:03.971804 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:03.971804 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:03.971804 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:03.971804 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:03.971804 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:03.972685 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:03.972685 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:03.972685 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:03.972685 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:03.974581 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 229/463] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:03.975647 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d15962-145b-46c2-ac52-8a7c3419900e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:03.994189 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b2af3-686b-491e-9b67-f24535c88c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:03.994878 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b2af3-686b-491e-9b67-f24535c88c1d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:03.995186 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b2af3-686b-491e-9b67-f24535c88c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:03.995464 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b2af3-686b-491e-9b67-f24535c88c1d None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:03.996207 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b2af3-686b-491e-9b67-f24535c88c1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:03.996630 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b2af3-686b-491e-9b67-f24535c88c1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:03.998124 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b2af3-686b-491e-9b67-f24535c88c1d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:04.008355 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:04.015891 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 235/464] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:04.015891 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b2af3-686b-491e-9b67-f24535c88c1d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:04.097060 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d15962-145b-46c2-ac52-8a7c3419900e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:04.128056 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b2af3-686b-491e-9b67-f24535c88c1d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:04.133195 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173abf73-efde-4c7b-9e8c-e8724f16fc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:04.134232 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173abf73-efde-4c7b-9e8c-e8724f16fc21 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:04.134734 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173abf73-efde-4c7b-9e8c-e8724f16fc21 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:04.135249 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173abf73-efde-4c7b-9e8c-e8724f16fc21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:04.150401 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173abf73-efde-4c7b-9e8c-e8724f16fc21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:04.152483 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173abf73-efde-4c7b-9e8c-e8724f16fc21 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:04.172159 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc44b2e-fa8e-4d97-abcb-564f1fce0634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:04.172832 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc44b2e-fa8e-4d97-abcb-564f1fce0634 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:04.173118 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc44b2e-fa8e-4d97-abcb-564f1fce0634 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:04.173399 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc44b2e-fa8e-4d97-abcb-564f1fce0634 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:04.174127 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc44b2e-fa8e-4d97-abcb-564f1fce0634 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:04.174437 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc44b2e-fa8e-4d97-abcb-564f1fce0634 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:04.175538 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-173abf73-efde-4c7b-9e8c-e8724f16fc21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:04.175538 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:04.175538 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:04.175538 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:04.175538 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:04.175538 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:04.175538 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:04.175538 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:04.175538 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:04.175538 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:04.175538 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:04.175538 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:04.175538 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:04.175538 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:04.175538 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:04.176356 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:04.176356 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:04.176356 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:04.176356 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:04.176356 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:04.176356 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:04.176356 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:04.176356 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:04.176356 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:04.176356 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:04.176356 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:04.176356 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:04.176356 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:04.176356 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:04.176356 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:04.176356 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:04.176356 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:04.177208 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:04.177208 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:04.177208 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:04.177208 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:04.177415 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc44b2e-fa8e-4d97-abcb-564f1fce0634 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:04.180140 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 230/465] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:04.181278 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173abf73-efde-4c7b-9e8c-e8724f16fc21 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:04.186481 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:04.191117 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 236/466] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:04.192245 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc44b2e-fa8e-4d97-abcb-564f1fce0634 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:04.273135 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:04.305878 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173abf73-efde-4c7b-9e8c-e8724f16fc21 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:04.306071 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc44b2e-fa8e-4d97-abcb-564f1fce0634 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:04.341812 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e18f35-2446-4258-9d63-302d83f8063a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:04.342479 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e18f35-2446-4258-9d63-302d83f8063a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:04.342889 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e18f35-2446-4258-9d63-302d83f8063a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:04.343278 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e18f35-2446-4258-9d63-302d83f8063a None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:04.344041 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e18f35-2446-4258-9d63-302d83f8063a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:04.344349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e18f35-2446-4258-9d63-302d83f8063a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:04.344458 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4939b086-8102-452b-889a-6cad564794bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:04.345103 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939b086-8102-452b-889a-6cad564794bd None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:04.345391 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939b086-8102-452b-889a-6cad564794bd None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:04.345663 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4939b086-8102-452b-889a-6cad564794bd None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:04.345770 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e18f35-2446-4258-9d63-302d83f8063a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:04.346366 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4939b086-8102-452b-889a-6cad564794bd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:04.348469 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4939b086-8102-452b-889a-6cad564794bd None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:04.357291 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 237/467] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:04.358025 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e18f35-2446-4258-9d63-302d83f8063a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:04.370568 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 231/468] 10.222.0.50 () {60 vars in 1132 bytes} [Fri May 27 15:01:04 2022] POST /identity/v3/users => generated 246 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:01:04.371929 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ff9b0-051e-466b-848c-206f0f0b28dd None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:04.372211 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ff9b0-051e-466b-848c-206f0f0b28dd None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:04.372489 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ff9b0-051e-466b-848c-206f0f0b28dd None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:04.373916 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 232/469] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:04.374442 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ff9b0-051e-466b-848c-206f0f0b28dd None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:04.445795 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:04.504493 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ff9b0-051e-466b-848c-206f0f0b28dd None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:04.515988 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e18f35-2446-4258-9d63-302d83f8063a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:04.538499 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:04.551240 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbec0cb-0a6c-498d-8952-6f7afece7670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:04.551927 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbec0cb-0a6c-498d-8952-6f7afece7670 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:04.552211 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbec0cb-0a6c-498d-8952-6f7afece7670 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:04.552486 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbec0cb-0a6c-498d-8952-6f7afece7670 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:04.553201 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbec0cb-0a6c-498d-8952-6f7afece7670 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:04.554683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbec0cb-0a6c-498d-8952-6f7afece7670 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:04.577829 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0239bc-5323-453d-8f2a-9ff57c731aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:04.579510 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0239bc-5323-453d-8f2a-9ff57c731aa0 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:04.580003 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0239bc-5323-453d-8f2a-9ff57c731aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:04.580429 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0239bc-5323-453d-8f2a-9ff57c731aa0 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:04.581314 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0239bc-5323-453d-8f2a-9ff57c731aa0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:04.582954 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0239bc-5323-453d-8f2a-9ff57c731aa0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:04.586991 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 233/470] 10.222.0.50 () {60 vars in 1132 bytes} [Fri May 27 15:01:04 2022] POST /identity/v3/users => generated 248 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:01:04.587731 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4c098-68ca-4085-9546-19e9d3251077 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:04.588284 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4c098-68ca-4085-9546-19e9d3251077 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:04.588563 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf4c098-68ca-4085-9546-19e9d3251077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:04.611592 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-fbf4c098-68ca-4085-9546-19e9d3251077 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:04.612818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fbf4c098-68ca-4085-9546-19e9d3251077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:04.613718 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 238/471] 10.222.0.50 () {60 vars in 1132 bytes} [Fri May 27 15:01:04 2022] POST /identity/v3/users => generated 248 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:01:04.696212 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 234/472] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:01:04.728685 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0239bc-5323-453d-8f2a-9ff57c731aa0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:04.855087 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0239bc-5323-453d-8f2a-9ff57c731aa0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:04.874675 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:04.894344 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aefeea-2634-45d0-8bc7-00f90c2b466f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:04.895177 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aefeea-2634-45d0-8bc7-00f90c2b466f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:04.895475 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aefeea-2634-45d0-8bc7-00f90c2b466f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:04.895735 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aefeea-2634-45d0-8bc7-00f90c2b466f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:04.907555 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aefeea-2634-45d0-8bc7-00f90c2b466f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:04.909071 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aefeea-2634-45d0-8bc7-00f90c2b466f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:04.928850 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-37aefeea-2634-45d0-8bc7-00f90c2b466f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:04.928850 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:04.928850 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:04.928850 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:04.928850 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:04.928850 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:04.928850 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:04.928850 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:04.928850 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:04.928850 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:04.928850 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:04.928850 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:04.928850 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:04.928850 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:04.928850 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:04.929766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:04.929766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:04.929766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:04.929766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:04.929766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:04.929766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:04.929766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:04.929766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:04.929766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:04.929766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:04.929766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:04.929766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:04.929766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:04.929766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:04.929766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:04.929766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:04.929766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:04.930617 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:04.930617 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:04.930617 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:04.930617 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:04.930617 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 239/473] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:04.937767 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4c098-68ca-4085-9546-19e9d3251077 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:05.044455 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c79a28-4c52-430d-85ed-aea20fa3dbd2 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:05.044775 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c79a28-4c52-430d-85ed-aea20fa3dbd2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:05.045016 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c79a28-4c52-430d-85ed-aea20fa3dbd2 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:05.047219 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 240/474] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:05.052556 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee2c09-013b-4a3f-b3a1-b3d2bece9a77 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:05.052841 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee2c09-013b-4a3f-b3a1-b3d2bece9a77 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:05.053119 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee2c09-013b-4a3f-b3a1-b3d2bece9a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:05.062697 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf4c098-68ca-4085-9546-19e9d3251077 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:05.072498 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5aee2c09-013b-4a3f-b3a1-b3d2bece9a77 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:05.074147 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5aee2c09-013b-4a3f-b3a1-b3d2bece9a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:05.082810 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:05.102719 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e1f68d-7018-4436-b0e8-d6c1bcb2a07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:05.103529 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e1f68d-7018-4436-b0e8-d6c1bcb2a07c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:05.103884 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e1f68d-7018-4436-b0e8-d6c1bcb2a07c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:05.104235 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e1f68d-7018-4436-b0e8-d6c1bcb2a07c None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:05.105203 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e1f68d-7018-4436-b0e8-d6c1bcb2a07c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:05.105546 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e1f68d-7018-4436-b0e8-d6c1bcb2a07c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:05.107065 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e1f68d-7018-4436-b0e8-d6c1bcb2a07c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:05.124458 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 235/475] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:05.131015 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e1f68d-7018-4436-b0e8-d6c1bcb2a07c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:05.197175 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 241/476] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 15:01:05.219925 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aee2c09-013b-4a3f-b3a1-b3d2bece9a77 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:05.248002 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e1f68d-7018-4436-b0e8-d6c1bcb2a07c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:05.285430 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0915791c-9c86-4e88-b553-edc2eb03b11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:05.286447 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0915791c-9c86-4e88-b553-edc2eb03b11e None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:05.286948 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0915791c-9c86-4e88-b553-edc2eb03b11e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:05.287419 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0915791c-9c86-4e88-b553-edc2eb03b11e None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:05.288322 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0915791c-9c86-4e88-b553-edc2eb03b11e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:05.289931 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0915791c-9c86-4e88-b553-edc2eb03b11e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:05.319220 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 236/477] 10.222.0.50 () {60 vars in 1132 bytes} [Fri May 27 15:01:05 2022] POST /identity/v3/users => generated 249 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:01:05.444541 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:05.448889 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aee2c09-013b-4a3f-b3a1-b3d2bece9a77 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:05.501444 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c1b0bb-15d9-430c-b747-12d00620fc40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:05.502138 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1b0bb-15d9-430c-b747-12d00620fc40 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:05.502424 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1b0bb-15d9-430c-b747-12d00620fc40 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:05.502703 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c1b0bb-15d9-430c-b747-12d00620fc40 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:05.503509 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c1b0bb-15d9-430c-b747-12d00620fc40 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:05.504909 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c1b0bb-15d9-430c-b747-12d00620fc40 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:05.517768 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.sql.core [None req-91c1b0bb-15d9-430c-b747-12d00620fc40 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 27 15:01:05.517768 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 27 15:01:05.517768 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [parameters: {'id': '68dcb5f9eb964bb4ac5abc4ff807115c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 27 15:01:05.517768 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80395) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 27 15:01:05.534577 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-91c1b0bb-15d9-430c-b747-12d00620fc40 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 27 15:01:05.534577 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:05.534577 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 15:01:05.534577 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 15:01:05.534577 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 15:01:05.534577 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 15:01:05.534577 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 15:01:05.534577 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result = self._query(query) May 27 15:01:05.534577 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 15:01:05.534577 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application conn.query(q) May 27 15:01:05.534577 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 15:01:05.534577 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 15:01:05.534577 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 15:01:05.534577 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result.read() May 27 15:01:05.534577 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 15:01:05.535600 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 15:01:05.535600 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 15:01:05.535600 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 15:01:05.535600 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 15:01:05.535600 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 15:01:05.535600 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 15:01:05.535600 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 15:01:05.535600 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 27 15:01:05.535600 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:05.535600 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 27 15:01:05.535600 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:05.535600 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:05.535600 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 27 15:01:05.535600 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 27 15:01:05.535600 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 27 15:01:05.535600 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return ref.to_dict() May 27 15:01:05.535600 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 15:01:05.535600 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application next(self.gen) May 27 15:01:05.536547 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 27 15:01:05.536547 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application yield resource May 27 15:01:05.536547 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 15:01:05.536547 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application next(self.gen) May 27 15:01:05.536547 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 27 15:01:05.536547 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.session.rollback() May 27 15:01:05.536547 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 27 15:01:05.536547 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.force_reraise() May 27 15:01:05.536547 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 27 15:01:05.536547 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise self.value May 27 15:01:05.536547 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 27 15:01:05.536547 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 27 15:01:05.536547 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 27 15:01:05.536547 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application session.commit() May 27 15:01:05.536547 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 27 15:01:05.536547 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 27 15:01:05.536547 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 27 15:01:05.537410 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._prepare_impl() May 27 15:01:05.537410 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 27 15:01:05.537410 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.session.flush() May 27 15:01:05.537410 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 27 15:01:05.537410 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._flush(objects) May 27 15:01:05.537410 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 27 15:01:05.537410 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 27 15:01:05.537410 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 27 15:01:05.537410 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application compat.raise_( May 27 15:01:05.537410 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 15:01:05.537410 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception May 27 15:01:05.537410 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 27 15:01:05.537410 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application flush_context.execute() May 27 15:01:05.537410 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 27 15:01:05.537410 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rec.execute(self) May 27 15:01:05.537410 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 27 15:01:05.537410 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 27 15:01:05.538298 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 27 15:01:05.538298 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application _emit_insert_statements( May 27 15:01:05.538298 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 27 15:01:05.538298 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application c = connection._execute_20( May 27 15:01:05.538298 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 27 15:01:05.538298 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 27 15:01:05.538298 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 27 15:01:05.538298 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 27 15:01:05.538298 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 27 15:01:05.538298 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ret = self._execute_context( May 27 15:01:05.538298 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 27 15:01:05.538298 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 27 15:01:05.538298 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 27 15:01:05.538298 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 27 15:01:05.538298 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 15:01:05.538298 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception May 27 15:01:05.540776 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 15:01:05.540776 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 15:01:05.540776 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 15:01:05.540776 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 15:01:05.540776 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 15:01:05.540776 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result = self._query(query) May 27 15:01:05.540776 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 15:01:05.540776 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application conn.query(q) May 27 15:01:05.540776 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 15:01:05.540776 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 15:01:05.540776 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 15:01:05.540776 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result.read() May 27 15:01:05.540776 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 15:01:05.540776 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 15:01:05.540776 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 15:01:05.540776 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 15:01:05.542688 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 15:01:05.542688 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 15:01:05.542688 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 15:01:05.542688 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 15:01:05.542688 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 27 15:01:05.542688 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 27 15:01:05.542688 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [parameters: {'id': '68dcb5f9eb964bb4ac5abc4ff807115c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 27 15:01:05.542688 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 27 15:01:05.542688 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:05.542688 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 15:01:05.542688 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:05.542688 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:05.542688 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:05.542688 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:05.542688 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:05.542688 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:05.551456 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:05.551456 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:05.551456 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:05.551456 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:05.551456 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:05.551456 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:05.551456 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 27 15:01:05.551456 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 27 15:01:05.551456 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:05.551456 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:05.551456 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 27 15:01:05.551456 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 27 15:01:05.551456 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 27 15:01:05.551456 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 27 15:01:05.551456 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 27 15:01:05.551456 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:05.553194 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 242/478] 10.222.0.50 () {60 vars in 1131 bytes} [Fri May 27 15:01:05 2022] POST /identity/v3/roles => generated 144 bytes in 317 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 27 15:01:05.553305 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0915791c-9c86-4e88-b553-edc2eb03b11e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:05.596069 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:05.666565 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0915791c-9c86-4e88-b553-edc2eb03b11e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:05.701879 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49970025-9936-49a2-92ab-22be05766b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:05.703623 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49970025-9936-49a2-92ab-22be05766b75 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:05.704110 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49970025-9936-49a2-92ab-22be05766b75 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:05.704625 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49970025-9936-49a2-92ab-22be05766b75 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:05.716562 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49970025-9936-49a2-92ab-22be05766b75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:01:05.726472 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-49970025-9936-49a2-92ab-22be05766b75 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 15:01:05.726472 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:05.726472 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:05.726472 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:05.726472 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:05.726472 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:05.726472 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:05.726472 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:05.726472 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:05.726472 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:05.726472 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:05.726472 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:05.726472 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:01:05.726472 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:01:05.726472 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:01:05.726472 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:01:05.727416 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:01:05.727416 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:01:05.727416 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:01:05.727416 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:01:05.727416 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:05.727416 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:05.727416 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:05.727416 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:05.727416 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:05.727416 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:05.727416 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:05.727416 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:05.727416 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:05.727416 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:05.727416 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:05.727416 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:05.728215 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:05.728215 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:05.728215 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:05.728215 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:05.728215 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:01:05.728215 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:01:05.728215 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:01:05.728215 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:01:05.728215 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:01:05.728215 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:01:05.728215 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 27 15:01:05.728215 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:05.730650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 237/479] 10.222.0.50 () {56 vars in 1114 bytes} [Fri May 27 15:01:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:05.740286 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c1b0bb-15d9-430c-b747-12d00620fc40 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:05.856555 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c1b0bb-15d9-430c-b747-12d00620fc40 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:05.896208 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f562cc-1db4-4a18-a2a3-3c9ae8e69779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:05.897230 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f562cc-1db4-4a18-a2a3-3c9ae8e69779 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:05.897684 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f562cc-1db4-4a18-a2a3-3c9ae8e69779 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:05.898130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f562cc-1db4-4a18-a2a3-3c9ae8e69779 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:05.899069 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f562cc-1db4-4a18-a2a3-3c9ae8e69779 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:05.899562 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f562cc-1db4-4a18-a2a3-3c9ae8e69779 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:05.901089 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f562cc-1db4-4a18-a2a3-3c9ae8e69779 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:05.910961 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 243/480] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:01:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:07.172578 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f71e3e-55c0-485e-a83e-25b9326c5c45 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:07.173109 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f71e3e-55c0-485e-a83e-25b9326c5c45 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:07.173109 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f71e3e-55c0-485e-a83e-25b9326c5c45 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:07.174789 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 238/481] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:07.180170 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa0947-7c28-4b8a-9aca-735e0c598191 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:07.180593 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa0947-7c28-4b8a-9aca-735e0c598191 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:07.180763 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa0947-7c28-4b8a-9aca-735e0c598191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:07.202141 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-41fa0947-7c28-4b8a-9aca-735e0c598191 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:07.203927 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-41fa0947-7c28-4b8a-9aca-735e0c598191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:07.225019 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:07.305462 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 244/482] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:01:07.323275 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd056745-a5ed-4d36-ad0f-c4e6ce9040d3 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:07.323862 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd056745-a5ed-4d36-ad0f-c4e6ce9040d3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:07.324333 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd056745-a5ed-4d36-ad0f-c4e6ce9040d3 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:07.326126 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 239/483] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:07.332257 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d1254-ade4-4fea-9fc4-0a90c1de330b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:07.332817 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d1254-ade4-4fea-9fc4-0a90c1de330b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:07.333298 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d1254-ade4-4fea-9fc4-0a90c1de330b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:07.348470 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd056745-a5ed-4d36-ad0f-c4e6ce9040d3 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:07.362882 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c24d1254-ade4-4fea-9fc4-0a90c1de330b None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:07.365589 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c24d1254-ade4-4fea-9fc4-0a90c1de330b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:07.481919 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 245/484] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:01:07.488770 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69f64d-e0f0-419a-a9ea-e2c72e32ee4a None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:07.488770 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69f64d-e0f0-419a-a9ea-e2c72e32ee4a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:07.488770 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69f64d-e0f0-419a-a9ea-e2c72e32ee4a None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:07.489249 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 246/485] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:07.490562 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20532fd-5d7d-4979-b1da-a4665ceb1fa1 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:07.491243 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20532fd-5d7d-4979-b1da-a4665ceb1fa1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:07.491436 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20532fd-5d7d-4979-b1da-a4665ceb1fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:07.500912 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:07.511495 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f20532fd-5d7d-4979-b1da-a4665ceb1fa1 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:07.512807 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f20532fd-5d7d-4979-b1da-a4665ceb1fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:07.513054 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd056745-a5ed-4d36-ad0f-c4e6ce9040d3 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:07.566230 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5208c973-1ff4-423b-97fc-9be29b4e7a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:07.567339 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208c973-1ff4-423b-97fc-9be29b4e7a28 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:07.567759 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208c973-1ff4-423b-97fc-9be29b4e7a28 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:07.568348 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5208c973-1ff4-423b-97fc-9be29b4e7a28 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:07.582750 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5208c973-1ff4-423b-97fc-9be29b4e7a28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:01:07.593965 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-5208c973-1ff4-423b-97fc-9be29b4e7a28 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 15:01:07.593965 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:07.593965 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:07.593965 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:07.593965 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:07.593965 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:07.593965 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:07.593965 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:07.593965 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:07.593965 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:07.593965 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:07.593965 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:07.593965 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:01:07.593965 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:01:07.593965 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:01:07.593965 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:01:07.594942 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:01:07.594942 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:01:07.594942 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:01:07.594942 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:01:07.594942 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:07.594942 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:07.594942 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:07.594942 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:07.594942 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:07.594942 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:07.594942 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:07.594942 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:07.594942 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:07.594942 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:07.594942 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:07.594942 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:07.595746 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:07.595746 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:07.595746 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:07.595746 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:07.595746 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:01:07.595746 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:01:07.595746 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:01:07.595746 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:01:07.595746 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:01:07.595746 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:01:07.595746 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 15:01:07.595746 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:07.597043 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 240/486] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 15:01:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:07.599472 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5208c973-1ff4-423b-97fc-9be29b4e7a28 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:07.610992 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 247/487] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:01:07.612129 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20532fd-5d7d-4979-b1da-a4665ceb1fa1 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:07.643623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:07.757434 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5208c973-1ff4-423b-97fc-9be29b4e7a28 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:07.758505 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20532fd-5d7d-4979-b1da-a4665ceb1fa1 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:07.796237 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c62897-68e3-42af-a577-7c12530acb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:07.797559 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c62897-68e3-42af-a577-7c12530acb2d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:07.797845 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c62897-68e3-42af-a577-7c12530acb2d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:07.798122 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c62897-68e3-42af-a577-7c12530acb2d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:07.805569 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90698e-de3f-42d0-a4f6-e5b5b69882c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:07.806922 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90698e-de3f-42d0-a4f6-e5b5b69882c5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:07.807428 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90698e-de3f-42d0-a4f6-e5b5b69882c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:07.807903 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b90698e-de3f-42d0-a4f6-e5b5b69882c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:07.808935 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c62897-68e3-42af-a577-7c12530acb2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:01:07.809333 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90698e-de3f-42d0-a4f6-e5b5b69882c5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:07.809835 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90698e-de3f-42d0-a4f6-e5b5b69882c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:07.811416 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b90698e-de3f-42d0-a4f6-e5b5b69882c5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:07.817940 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-81c62897-68e3-42af-a577-7c12530acb2d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 15:01:07.817940 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:07.817940 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:07.817940 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:07.817940 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:07.817940 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:07.817940 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:07.817940 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:07.817940 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:07.817940 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:07.817940 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:07.817940 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:07.817940 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:01:07.817940 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:01:07.817940 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:01:07.817940 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:01:07.818882 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:01:07.818882 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:01:07.818882 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:01:07.818882 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:01:07.818882 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:07.818882 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:07.818882 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:07.818882 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:07.818882 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:07.818882 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:07.818882 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:07.818882 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:07.818882 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:07.818882 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:07.818882 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:07.818882 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:07.819691 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:07.819691 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:07.819691 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:07.819691 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:07.819691 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:01:07.819691 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:01:07.819691 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:01:07.819691 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:01:07.819691 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:01:07.819691 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:01:07.819691 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 15:01:07.819691 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:07.823553 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 241/488] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 15:01:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:07.824152 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 248/489] 10.222.0.50 () {56 vars in 1103 bytes} [Fri May 27 15:01:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:07.824406 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c62897-68e3-42af-a577-7c12530acb2d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:07.825707 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90698e-de3f-42d0-a4f6-e5b5b69882c5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:07.843236 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:07.940642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:07.952149 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c62897-68e3-42af-a577-7c12530acb2d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:07.975908 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b90698e-de3f-42d0-a4f6-e5b5b69882c5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:07.989829 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3116323-0e95-486d-b911-291970611e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:07.990621 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3116323-0e95-486d-b911-291970611e52 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:07.990946 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3116323-0e95-486d-b911-291970611e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:07.991232 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3116323-0e95-486d-b911-291970611e52 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:08.003068 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3116323-0e95-486d-b911-291970611e52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:01:08.010913 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-a3116323-0e95-486d-b911-291970611e52 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 15:01:08.010913 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:08.010913 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:08.010913 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:08.010913 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:08.010913 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:08.010913 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:08.010913 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:08.010913 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:08.010913 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:08.010913 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:08.010913 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:08.010913 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:01:08.010913 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:01:08.010913 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:01:08.010913 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:01:08.011805 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:01:08.011805 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:01:08.011805 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:01:08.011805 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:01:08.011805 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:08.011805 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:08.011805 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:08.011805 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:08.011805 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:08.011805 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:08.011805 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:08.011805 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:08.011805 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:08.011805 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:08.011805 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:08.011805 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:08.012617 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:08.012617 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:08.012617 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:08.012617 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:08.012617 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:01:08.012617 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:01:08.012617 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:01:08.012617 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:01:08.012617 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:01:08.012617 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:01:08.012617 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 15:01:08.012617 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:08.012617 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 242/490] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 15:01:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:08.013832 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3116323-0e95-486d-b911-291970611e52 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:08.024406 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5d2baf-7b8f-4ed1-8a1c-82e9ccf3ea10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:08.025819 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d2baf-7b8f-4ed1-8a1c-82e9ccf3ea10 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:08.026348 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d2baf-7b8f-4ed1-8a1c-82e9ccf3ea10 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:08.027389 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d2baf-7b8f-4ed1-8a1c-82e9ccf3ea10 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:08.028673 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5d2baf-7b8f-4ed1-8a1c-82e9ccf3ea10 None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:08.029161 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5d2baf-7b8f-4ed1-8a1c-82e9ccf3ea10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:08.031979 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5d2baf-7b8f-4ed1-8a1c-82e9ccf3ea10 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:08.046310 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 249/491] 10.222.0.50 () {56 vars in 1103 bytes} [Fri May 27 15:01:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:08.047901 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5d2baf-7b8f-4ed1-8a1c-82e9ccf3ea10 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:08.099510 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:08.127826 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3116323-0e95-486d-b911-291970611e52 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:08.182794 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5d2baf-7b8f-4ed1-8a1c-82e9ccf3ea10 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:08.183746 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba3c5a-e6a1-4467-a378-bece6966f5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:08.185236 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba3c5a-e6a1-4467-a378-bece6966f5d2 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:08.186650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba3c5a-e6a1-4467-a378-bece6966f5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:08.187177 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba3c5a-e6a1-4467-a378-bece6966f5d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:08.204545 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ba3c5a-e6a1-4467-a378-bece6966f5d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:08.206285 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ba3c5a-e6a1-4467-a378-bece6966f5d2 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:08.223955 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db8876-a58e-4c89-a755-eac7006addf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:08.225308 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db8876-a58e-4c89-a755-eac7006addf9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:08.225693 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db8876-a58e-4c89-a755-eac7006addf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:08.226070 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db8876-a58e-4c89-a755-eac7006addf9 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:08.228089 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3db8876-a58e-4c89-a755-eac7006addf9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:08.228602 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3db8876-a58e-4c89-a755-eac7006addf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:08.230206 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3db8876-a58e-4c89-a755-eac7006addf9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:08.234506 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-e3ba3c5a-e6a1-4467-a378-bece6966f5d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:08.234506 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:08.234506 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:08.234506 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:08.234506 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:08.234506 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:08.234506 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:08.234506 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:08.234506 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:08.234506 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:08.234506 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:08.234506 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:08.234506 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:08.234506 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:08.234506 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:08.235660 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:08.235660 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:08.235660 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:08.235660 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:08.235660 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:08.235660 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:08.235660 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:08.235660 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:08.235660 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:08.235660 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:08.235660 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:08.235660 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:08.235660 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:08.235660 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:08.235660 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:08.235660 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:08.235660 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:08.236740 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:08.236740 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:08.236740 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:08.236740 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:08.238609 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 243/492] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:08.239714 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:08.241032 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba3c5a-e6a1-4467-a378-bece6966f5d2 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:08.250697 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 250/493] 10.222.0.50 () {56 vars in 1103 bytes} [Fri May 27 15:01:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:08.251937 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db8876-a58e-4c89-a755-eac7006addf9 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:08.368540 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ba3c5a-e6a1-4467-a378-bece6966f5d2 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:08.379540 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db8876-a58e-4c89-a755-eac7006addf9 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:08.402747 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:08.413042 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bdb787-eefe-4923-b16a-af389779750e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:08.413836 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bdb787-eefe-4923-b16a-af389779750e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:08.414122 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bdb787-eefe-4923-b16a-af389779750e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:08.414396 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bdb787-eefe-4923-b16a-af389779750e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:08.416699 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd08e5-38aa-43a5-927e-8302fbf48258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:08.417355 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd08e5-38aa-43a5-927e-8302fbf48258 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:08.417638 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd08e5-38aa-43a5-927e-8302fbf48258 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:08.417913 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd08e5-38aa-43a5-927e-8302fbf48258 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:08.418631 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbd08e5-38aa-43a5-927e-8302fbf48258 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:08.418964 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbd08e5-38aa-43a5-927e-8302fbf48258 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:08.420356 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbd08e5-38aa-43a5-927e-8302fbf48258 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:08.425979 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bdb787-eefe-4923-b16a-af389779750e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:08.427485 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bdb787-eefe-4923-b16a-af389779750e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:08.432623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 251/494] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:08.433663 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd08e5-38aa-43a5-927e-8302fbf48258 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:08.446632 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-21bdb787-eefe-4923-b16a-af389779750e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:08.446632 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:08.446632 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:08.446632 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:08.446632 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:08.446632 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:08.446632 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:08.446632 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:08.446632 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:08.446632 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:08.446632 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:08.446632 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:08.446632 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:08.446632 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:08.446632 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:08.447572 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:08.447572 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:08.447572 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:08.447572 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:08.447572 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:08.447572 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:08.447572 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:08.447572 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:08.447572 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:08.447572 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:08.447572 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:08.447572 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:08.447572 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:08.447572 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:08.447572 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:08.447572 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:08.447572 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:08.448430 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:08.448430 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:08.448430 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:08.448430 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:08.448430 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 244/495] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:08.449187 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31229d5-6337-4ef1-a14b-0420225b6f37 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:08.449362 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31229d5-6337-4ef1-a14b-0420225b6f37 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:08.449790 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31229d5-6337-4ef1-a14b-0420225b6f37 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:08.455029 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 245/496] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:08.455029 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31229d5-6337-4ef1-a14b-0420225b6f37 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:08.504110 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:08.550319 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd08e5-38aa-43a5-927e-8302fbf48258 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:08.566631 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31229d5-6337-4ef1-a14b-0420225b6f37 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:08.585336 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397c41a6-8458-441d-bb38-275b3c160a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:08.586127 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c41a6-8458-441d-bb38-275b3c160a66 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:08.586406 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c41a6-8458-441d-bb38-275b3c160a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:08.586676 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c41a6-8458-441d-bb38-275b3c160a66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:08.598983 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397c41a6-8458-441d-bb38-275b3c160a66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:08.600597 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397c41a6-8458-441d-bb38-275b3c160a66 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:08.603181 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c62a78-8b4f-4484-91fc-de22e805afb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:08.603967 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c62a78-8b4f-4484-91fc-de22e805afb4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:08.604305 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c62a78-8b4f-4484-91fc-de22e805afb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:08.604586 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c62a78-8b4f-4484-91fc-de22e805afb4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:08.613217 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c62a78-8b4f-4484-91fc-de22e805afb4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:08.614675 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c62a78-8b4f-4484-91fc-de22e805afb4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:08.619146 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-397c41a6-8458-441d-bb38-275b3c160a66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:08.619146 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:08.619146 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:08.619146 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:08.619146 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:08.619146 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:08.619146 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:08.619146 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:08.619146 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:08.619146 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:08.619146 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:08.619146 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:08.619146 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:08.619146 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:08.619146 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:08.620021 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:08.620021 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:08.620021 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:08.620021 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:08.620021 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:08.620021 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:08.620021 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:08.620021 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:08.620021 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:08.620021 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:08.620021 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:08.620021 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:08.620021 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:08.620021 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:08.620021 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:08.620021 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:08.620021 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:08.626974 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:08.626974 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:08.626974 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:08.626974 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:08.626974 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 252/497] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:08.626974 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397c41a6-8458-441d-bb38-275b3c160a66 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:08.636307 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-91c62a78-8b4f-4484-91fc-de22e805afb4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 15:01:08.636307 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:08.636307 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:08.636307 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:08.636307 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:08.636307 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:08.636307 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:08.636307 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:08.636307 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:08.636307 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:08.636307 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:08.636307 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:08.636307 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 15:01:08.636307 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 15:01:08.636307 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 15:01:08.636307 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 15:01:08.637287 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:08.637287 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:08.637287 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 15:01:08.637287 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:08.637287 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 15:01:08.637287 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:08.637287 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:08.637287 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:08.637287 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:08.637287 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:08.637287 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:08.637287 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:08.637287 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:08.637287 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:08.637287 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:08.637287 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:08.637287 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:08.637287 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:08.638214 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:08.638214 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:08.638214 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 15:01:08.638214 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 15:01:08.638214 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 15:01:08.638214 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 15:01:08.638214 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 15:01:08.638214 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 15:01:08.638214 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 15:01:08.638214 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:08.638214 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 246/498] 10.222.0.50 () {56 vars in 1093 bytes} [Fri May 27 15:01:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:08.639752 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659b80a-333f-4a96-b600-578af85ad704 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:08.639752 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659b80a-333f-4a96-b600-578af85ad704 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:08.639752 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659b80a-333f-4a96-b600-578af85ad704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:08.659208 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7659b80a-333f-4a96-b600-578af85ad704 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:08.660667 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7659b80a-333f-4a96-b600-578af85ad704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:08.694698 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:08.735131 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 247/499] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:01:08.735992 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7659b80a-333f-4a96-b600-578af85ad704 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:08.748079 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397c41a6-8458-441d-bb38-275b3c160a66 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:08.765082 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:08.789020 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da4390-d332-41f6-b00f-6f9e63ad3e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:08.789683 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da4390-d332-41f6-b00f-6f9e63ad3e41 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:08.789959 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da4390-d332-41f6-b00f-6f9e63ad3e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:08.790231 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da4390-d332-41f6-b00f-6f9e63ad3e41 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:08.791226 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9da4390-d332-41f6-b00f-6f9e63ad3e41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:08.791334 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9da4390-d332-41f6-b00f-6f9e63ad3e41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:08.792625 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9da4390-d332-41f6-b00f-6f9e63ad3e41 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:08.806527 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 253/500] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:08.807731 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da4390-d332-41f6-b00f-6f9e63ad3e41 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:08.844222 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7659b80a-333f-4a96-b600-578af85ad704 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:08.878124 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b97c1a-a813-4038-b419-6e47db02fc1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:08.878705 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b97c1a-a813-4038-b419-6e47db02fc1a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:08.879025 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b97c1a-a813-4038-b419-6e47db02fc1a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:08.879300 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b97c1a-a813-4038-b419-6e47db02fc1a None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:08.880046 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b97c1a-a813-4038-b419-6e47db02fc1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:08.880359 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b97c1a-a813-4038-b419-6e47db02fc1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:08.881750 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b97c1a-a813-4038-b419-6e47db02fc1a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:08.892885 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 248/501] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:08.893681 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b97c1a-a813-4038-b419-6e47db02fc1a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:08.920769 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da4390-d332-41f6-b00f-6f9e63ad3e41 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:08.924050 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:08.955404 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a759d6d4-cd9d-4d7e-86a1-0d3cbb94a901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:08.956065 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759d6d4-cd9d-4d7e-86a1-0d3cbb94a901 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:08.956342 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759d6d4-cd9d-4d7e-86a1-0d3cbb94a901 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:08.956611 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a759d6d4-cd9d-4d7e-86a1-0d3cbb94a901 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:08.957432 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a759d6d4-cd9d-4d7e-86a1-0d3cbb94a901 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:08.957734 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a759d6d4-cd9d-4d7e-86a1-0d3cbb94a901 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:08.959166 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a759d6d4-cd9d-4d7e-86a1-0d3cbb94a901 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:08.995671 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b97c1a-a813-4038-b419-6e47db02fc1a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:08.996515 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 254/502] 10.222.0.50 () {56 vars in 1133 bytes} [Fri May 27 15:01:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:08.997342 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a759d6d4-cd9d-4d7e-86a1-0d3cbb94a901 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:09.026302 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:09.033123 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5bcf98-b82b-4be2-aeb0-ca072bd858c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:09.033906 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5bcf98-b82b-4be2-aeb0-ca072bd858c4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:09.034183 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5bcf98-b82b-4be2-aeb0-ca072bd858c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:09.034453 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5bcf98-b82b-4be2-aeb0-ca072bd858c4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:09.044607 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5bcf98-b82b-4be2-aeb0-ca072bd858c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:01:09.053179 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-3d5bcf98-b82b-4be2-aeb0-ca072bd858c4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 15:01:09.053179 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:09.053179 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:09.053179 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:09.053179 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:09.053179 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:09.053179 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:09.053179 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:09.053179 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:09.053179 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:09.053179 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:09.053179 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:09.053179 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:01:09.053179 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:01:09.053179 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:01:09.053179 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:01:09.054243 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:01:09.054243 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:01:09.054243 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:01:09.054243 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:01:09.054243 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:09.054243 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:09.054243 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:09.054243 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:09.054243 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:09.054243 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:09.054243 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:09.054243 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:09.054243 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:09.054243 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:09.054243 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:09.054243 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:09.055315 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:09.055315 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:09.055315 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:09.055315 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:09.055315 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:01:09.055315 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:01:09.055315 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:01:09.055315 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:01:09.055315 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:01:09.055315 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:01:09.055315 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 15:01:09.055315 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:09.055315 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 249/503] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 15:01:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:09.056119 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ca349-7a59-417a-a19e-ded2c62df50f None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:09.056119 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ca349-7a59-417a-a19e-ded2c62df50f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:09.056269 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ca349-7a59-417a-a19e-ded2c62df50f None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:09.057412 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 250/504] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:09.058988 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ca349-7a59-417a-a19e-ded2c62df50f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:09.103919 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a759d6d4-cd9d-4d7e-86a1-0d3cbb94a901 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:09.136487 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573e8d23-7983-42db-b727-8e1b2f3207ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:09.137207 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e8d23-7983-42db-b727-8e1b2f3207ba None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:09.137484 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e8d23-7983-42db-b727-8e1b2f3207ba None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:09.137756 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573e8d23-7983-42db-b727-8e1b2f3207ba None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:09.144841 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573e8d23-7983-42db-b727-8e1b2f3207ba None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:09.146274 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573e8d23-7983-42db-b727-8e1b2f3207ba None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:09.147898 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:09.163901 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ca349-7a59-417a-a19e-ded2c62df50f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:09.165724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-573e8d23-7983-42db-b727-8e1b2f3207ba None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 27 15:01:09.165724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:09.165724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:09.165724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:09.165724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:09.165724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:09.165724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:09.165724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:09.165724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:09.165724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:09.165724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:09.165724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:09.165724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 15:01:09.165724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 15:01:09.165724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 15:01:09.165724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 15:01:09.166592 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:09.166592 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:09.166592 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 15:01:09.166592 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:09.166592 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 15:01:09.166592 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:09.166592 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:09.166592 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:09.166592 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:09.166592 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:09.166592 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:09.166592 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:09.166592 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:09.166592 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:09.166592 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:09.166592 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:09.166592 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:09.166592 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:09.167539 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:09.167539 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:09.167539 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 15:01:09.167539 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 15:01:09.167539 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 15:01:09.167539 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 15:01:09.167539 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 15:01:09.167539 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 15:01:09.167539 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 27 15:01:09.167539 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:09.167539 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 255/505] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:01:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:09.168095 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573e8d23-7983-42db-b727-8e1b2f3207ba None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:09.198588 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e8cd6-dc8f-471f-888a-64513f45673c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:09.199409 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e8cd6-dc8f-471f-888a-64513f45673c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:09.199691 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e8cd6-dc8f-471f-888a-64513f45673c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:09.199968 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e8cd6-dc8f-471f-888a-64513f45673c None admin] PATH_INFO: `/v3/users/glance` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:09.207599 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614e8cd6-dc8f-471f-888a-64513f45673c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:09.209058 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614e8cd6-dc8f-471f-888a-64513f45673c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:09.224216 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-614e8cd6-dc8f-471f-888a-64513f45673c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 27 15:01:09.224216 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:09.224216 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:09.224216 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:09.224216 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:09.224216 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:09.224216 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:09.224216 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:09.224216 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:09.224216 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:09.224216 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:09.224216 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:09.224216 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 15:01:09.224216 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 15:01:09.224216 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 15:01:09.224216 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 15:01:09.225300 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:09.225300 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:09.225300 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 15:01:09.225300 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:09.225300 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 15:01:09.225300 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:09.225300 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:09.225300 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:09.225300 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:09.225300 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:09.225300 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:09.225300 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:09.225300 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:09.225300 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:09.225300 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:09.225300 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:09.225300 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:09.225300 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:09.226311 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:09.226311 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:09.226311 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 15:01:09.226311 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 15:01:09.226311 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 15:01:09.226311 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 15:01:09.226311 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 15:01:09.226311 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 15:01:09.226311 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 27 15:01:09.226311 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:09.226311 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 251/506] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:01:09 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:09.226311 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e8cd6-dc8f-471f-888a-64513f45673c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:09.266997 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:09.278307 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573e8d23-7983-42db-b727-8e1b2f3207ba None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:09.312693 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff53c665-116a-4c21-8f38-839205a949ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:09.313343 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53c665-116a-4c21-8f38-839205a949ed None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:09.313624 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53c665-116a-4c21-8f38-839205a949ed None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:09.313905 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff53c665-116a-4c21-8f38-839205a949ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:09.335378 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e8cd6-dc8f-471f-888a-64513f45673c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:09.369382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b427f0a5-2c64-410c-8a61-ef212b256aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:09.370130 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427f0a5-2c64-410c-8a61-ef212b256aec None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:09.370481 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427f0a5-2c64-410c-8a61-ef212b256aec None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:09.370824 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427f0a5-2c64-410c-8a61-ef212b256aec None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:09.371619 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b427f0a5-2c64-410c-8a61-ef212b256aec None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:09.371926 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b427f0a5-2c64-410c-8a61-ef212b256aec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:09.373306 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b427f0a5-2c64-410c-8a61-ef212b256aec None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:09.381012 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff53c665-116a-4c21-8f38-839205a949ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:09.381196 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 252/507] 10.222.0.50 () {56 vars in 1103 bytes} [Fri May 27 15:01:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:09.382440 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ede6e-a5f4-47e5-8fa7-80d91d14b4e5 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:09.382542 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff53c665-116a-4c21-8f38-839205a949ed None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:09.382920 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ede6e-a5f4-47e5-8fa7-80d91d14b4e5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:09.383037 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707ede6e-a5f4-47e5-8fa7-80d91d14b4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:09.399405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-707ede6e-a5f4-47e5-8fa7-80d91d14b4e5 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:09.400519 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-707ede6e-a5f4-47e5-8fa7-80d91d14b4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:09.403225 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:09.465494 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 253/508] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:01:09.466326 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707ede6e-a5f4-47e5-8fa7-80d91d14b4e5 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:09.482154 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:09.492583 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 256/509] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:01:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 15:01:09.493465 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff53c665-116a-4c21-8f38-839205a949ed None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:09.565161 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707ede6e-a5f4-47e5-8fa7-80d91d14b4e5 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:09.590069 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff53c665-116a-4c21-8f38-839205a949ed None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:09.600051 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d13f240-2ff7-460b-88af-328cd689ca5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:09.600728 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13f240-2ff7-460b-88af-328cd689ca5a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:09.601012 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13f240-2ff7-460b-88af-328cd689ca5a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:09.601292 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13f240-2ff7-460b-88af-328cd689ca5a None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:09.602566 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d13f240-2ff7-460b-88af-328cd689ca5a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:09.602913 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d13f240-2ff7-460b-88af-328cd689ca5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:09.604383 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d13f240-2ff7-460b-88af-328cd689ca5a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:09.625970 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b10e97d-aa76-4a9a-bec7-e7daf22c42f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:09.626638 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b10e97d-aa76-4a9a-bec7-e7daf22c42f2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:09.626954 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b10e97d-aa76-4a9a-bec7-e7daf22c42f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:09.627233 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b10e97d-aa76-4a9a-bec7-e7daf22c42f2 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:09.628223 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b10e97d-aa76-4a9a-bec7-e7daf22c42f2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:09.628793 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b10e97d-aa76-4a9a-bec7-e7daf22c42f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:09.631403 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b10e97d-aa76-4a9a-bec7-e7daf22c42f2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:09.636896 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:09.649961 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 257/510] 10.222.0.50 () {56 vars in 1137 bytes} [Fri May 27 15:01:09 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:09.651357 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 254/511] 10.222.0.50 () {56 vars in 1137 bytes} [Fri May 27 15:01:09 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:09.651495 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b10e97d-aa76-4a9a-bec7-e7daf22c42f2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:09.651584 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d13f240-2ff7-460b-88af-328cd689ca5a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:09.701162 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:09.748975 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d13f240-2ff7-460b-88af-328cd689ca5a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:09.752779 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b10e97d-aa76-4a9a-bec7-e7daf22c42f2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:09.784126 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2007152e-fee5-48e7-9024-e917316d277d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:09.784904 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2007152e-fee5-48e7-9024-e917316d277d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:09.785184 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2007152e-fee5-48e7-9024-e917316d277d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:09.785451 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2007152e-fee5-48e7-9024-e917316d277d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:09.786233 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2ce97-5117-441f-90be-c50a5508fbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:09.787043 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2ce97-5117-441f-90be-c50a5508fbf0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:09.787332 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2ce97-5117-441f-90be-c50a5508fbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:09.787606 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b2ce97-5117-441f-90be-c50a5508fbf0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:09.795182 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2007152e-fee5-48e7-9024-e917316d277d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:09.796685 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2007152e-fee5-48e7-9024-e917316d277d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:09.797467 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b2ce97-5117-441f-90be-c50a5508fbf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:09.798922 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b2ce97-5117-441f-90be-c50a5508fbf0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:09.814105 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-2007152e-fee5-48e7-9024-e917316d277d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:09.814105 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:09.814105 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:09.814105 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:09.814105 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:09.814105 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:09.814105 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:09.814105 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:09.814105 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:09.814105 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:09.814105 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:09.814105 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:09.814105 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:09.814105 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:09.814105 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:09.815814 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:09.815814 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:09.815814 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:09.815814 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:09.815814 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:09.815814 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:09.815814 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:09.815814 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:09.815814 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:09.815814 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:09.815814 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:09.815814 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:09.815814 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:09.815814 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:09.815814 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:09.815814 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:09.815814 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:09.817442 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:09.817442 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:09.817442 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:09.817442 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:09.817442 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 255/512] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:09.817442 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2007152e-fee5-48e7-9024-e917316d277d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:09.818053 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-a9b2ce97-5117-441f-90be-c50a5508fbf0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:09.818053 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:09.818053 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:09.818053 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:09.818053 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:09.818053 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:09.818053 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:09.818053 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:09.818053 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:09.818053 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:09.818053 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:09.818053 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:09.818053 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:09.818053 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:09.818053 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:09.819560 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:09.819560 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:09.819560 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:09.819560 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:09.819560 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:09.819560 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:09.819560 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:09.819560 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:09.819560 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:09.819560 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:09.819560 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:09.819560 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:09.819560 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:09.819560 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:09.819560 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:09.819560 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:09.819560 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:09.821152 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:09.821152 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:09.821152 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:09.821152 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:09.821152 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 258/513] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:09.821152 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2ce97-5117-441f-90be-c50a5508fbf0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:09.872099 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:09.919333 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:09.924513 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b2ce97-5117-441f-90be-c50a5508fbf0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:09.928049 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2007152e-fee5-48e7-9024-e917316d277d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:09.963835 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1bbe2e-4e23-4fb2-becd-cdf170ad3454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:09.964888 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1bbe2e-4e23-4fb2-becd-cdf170ad3454 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:09.964998 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5228fa-a30b-4493-a50b-d1b02fe3c5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:09.965174 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1bbe2e-4e23-4fb2-becd-cdf170ad3454 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:09.965644 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1bbe2e-4e23-4fb2-becd-cdf170ad3454 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:09.965742 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5228fa-a30b-4493-a50b-d1b02fe3c5fd None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:09.965978 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5228fa-a30b-4493-a50b-d1b02fe3c5fd None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:09.966256 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5228fa-a30b-4493-a50b-d1b02fe3c5fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:09.976512 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5228fa-a30b-4493-a50b-d1b02fe3c5fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:09.977968 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5228fa-a30b-4493-a50b-d1b02fe3c5fd None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:09.996066 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-cb5228fa-a30b-4493-a50b-d1b02fe3c5fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:09.996066 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:09.996066 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:09.996066 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:09.996066 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:09.996066 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:09.996066 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:09.996066 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:09.996066 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:09.996066 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:09.996066 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:09.996066 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:09.996066 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:09.996066 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:09.996066 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:09.996932 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:09.996932 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:09.996932 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:09.996932 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:09.996932 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:09.996932 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:09.996932 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:09.996932 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:09.996932 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:09.996932 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:09.996932 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:09.996932 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:09.996932 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:09.996932 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:09.996932 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:09.996932 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:09.996932 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:09.997790 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:09.997790 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:09.997790 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:09.997790 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:09.997790 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 256/514] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:09.998060 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5228fa-a30b-4493-a50b-d1b02fe3c5fd None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:10.034563 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1bbe2e-4e23-4fb2-becd-cdf170ad3454 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:10.036058 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1bbe2e-4e23-4fb2-becd-cdf170ad3454 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:10.096543 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5228fa-a30b-4493-a50b-d1b02fe3c5fd None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:10.098015 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:10.129002 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:10.131902 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad91dd61-65f1-4eec-a000-cad7cb15aa5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:10.132546 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad91dd61-65f1-4eec-a000-cad7cb15aa5a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:10.132832 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad91dd61-65f1-4eec-a000-cad7cb15aa5a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:10.133110 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad91dd61-65f1-4eec-a000-cad7cb15aa5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:10.137521 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 259/515] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:01:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 15:01:10.138594 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1bbe2e-4e23-4fb2-becd-cdf170ad3454 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:10.206517 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad91dd61-65f1-4eec-a000-cad7cb15aa5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:10.207960 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad91dd61-65f1-4eec-a000-cad7cb15aa5a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:10.236973 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1bbe2e-4e23-4fb2-becd-cdf170ad3454 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:10.271985 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dd265e-3bc2-4389-beee-f66fc9c61944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:10.273079 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd265e-3bc2-4389-beee-f66fc9c61944 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:10.273534 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd265e-3bc2-4389-beee-f66fc9c61944 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:10.273982 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd265e-3bc2-4389-beee-f66fc9c61944 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:10.275080 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dd265e-3bc2-4389-beee-f66fc9c61944 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:10.276307 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dd265e-3bc2-4389-beee-f66fc9c61944 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:10.276753 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dd265e-3bc2-4389-beee-f66fc9c61944 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:10.291875 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 260/516] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:10.292944 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dd265e-3bc2-4389-beee-f66fc9c61944 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:10.328252 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 257/517] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:01:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 15:01:10.329116 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad91dd61-65f1-4eec-a000-cad7cb15aa5a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:10.331412 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:10.352682 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:10.396116 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dd265e-3bc2-4389-beee-f66fc9c61944 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:10.428704 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad91dd61-65f1-4eec-a000-cad7cb15aa5a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:10.429518 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f1c3b1-50d9-406d-857e-c28aa6659652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:10.430176 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f1c3b1-50d9-406d-857e-c28aa6659652 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:10.430454 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f1c3b1-50d9-406d-857e-c28aa6659652 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:10.430733 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f1c3b1-50d9-406d-857e-c28aa6659652 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:10.431480 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f1c3b1-50d9-406d-857e-c28aa6659652 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:10.431779 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f1c3b1-50d9-406d-857e-c28aa6659652 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:10.433138 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f1c3b1-50d9-406d-857e-c28aa6659652 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:10.444709 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 261/518] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:10.445484 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f1c3b1-50d9-406d-857e-c28aa6659652 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:10.461944 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f402e0cc-c7ad-4551-ad86-743cb6a1f26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:10.462531 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f402e0cc-c7ad-4551-ad86-743cb6a1f26b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:10.462815 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f402e0cc-c7ad-4551-ad86-743cb6a1f26b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:10.463371 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f402e0cc-c7ad-4551-ad86-743cb6a1f26b None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:10.464090 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f402e0cc-c7ad-4551-ad86-743cb6a1f26b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:10.464402 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f402e0cc-c7ad-4551-ad86-743cb6a1f26b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:10.465786 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f402e0cc-c7ad-4551-ad86-743cb6a1f26b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:10.477155 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 258/519] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:10.477939 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f402e0cc-c7ad-4551-ad86-743cb6a1f26b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:10.538271 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:10.549310 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f1c3b1-50d9-406d-857e-c28aa6659652 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:10.569430 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:10.581435 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f402e0cc-c7ad-4551-ad86-743cb6a1f26b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:10.584100 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc2220-b0eb-48b1-8e4f-7f492366b996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:10.584907 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc2220-b0eb-48b1-8e4f-7f492366b996 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:10.585192 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc2220-b0eb-48b1-8e4f-7f492366b996 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:10.585473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc2220-b0eb-48b1-8e4f-7f492366b996 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:10.597006 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdc2220-b0eb-48b1-8e4f-7f492366b996 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:10.598474 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdc2220-b0eb-48b1-8e4f-7f492366b996 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:10.618650 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-1cdc2220-b0eb-48b1-8e4f-7f492366b996 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:10.618650 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:10.618650 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:10.618650 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:10.618650 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:10.618650 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:10.618650 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:10.618650 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:10.618650 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:10.618650 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:10.618650 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:10.618650 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:10.618650 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:10.618650 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:10.618650 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:10.619610 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:10.619610 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:10.619610 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:10.619610 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:10.619610 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:10.619610 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:10.619610 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:10.619610 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:10.619610 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:10.619610 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:10.619610 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:10.619610 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:10.619610 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:10.619610 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:10.619610 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:10.619610 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:10.619610 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:10.620510 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:10.620510 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:10.620510 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:10.620510 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:10.620510 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 262/520] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:10.620782 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc2220-b0eb-48b1-8e4f-7f492366b996 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:10.627043 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b17f406-fb0b-4941-8996-1f264a5b08a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:10.627844 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17f406-fb0b-4941-8996-1f264a5b08a0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:10.628129 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17f406-fb0b-4941-8996-1f264a5b08a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:10.628403 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17f406-fb0b-4941-8996-1f264a5b08a0 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:10.638627 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b17f406-fb0b-4941-8996-1f264a5b08a0 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:10.641003 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b17f406-fb0b-4941-8996-1f264a5b08a0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:10.655592 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-6b17f406-fb0b-4941-8996-1f264a5b08a0 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 27 15:01:10.655592 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:10.655592 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:10.655592 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:10.655592 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:10.655592 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:10.655592 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:10.655592 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:10.655592 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:10.655592 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:10.655592 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:10.655592 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:10.655592 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 15:01:10.655592 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 15:01:10.655592 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 15:01:10.655592 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 15:01:10.657137 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:10.657137 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:10.657137 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 15:01:10.657137 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:10.657137 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 15:01:10.657137 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:10.657137 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:10.657137 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:10.657137 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:10.657137 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:10.657137 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:10.657137 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:10.657137 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:10.657137 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:10.657137 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:10.657137 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:10.657137 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:10.657137 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:10.658168 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:10.658168 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:10.658168 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 15:01:10.658168 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 15:01:10.658168 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 15:01:10.658168 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 15:01:10.658168 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 15:01:10.658168 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 15:01:10.658168 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 27 15:01:10.658168 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:10.658168 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 259/521] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 15:01:10 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:10.658864 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b17f406-fb0b-4941-8996-1f264a5b08a0 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:10.726874 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc2220-b0eb-48b1-8e4f-7f492366b996 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:10.757886 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b17f406-fb0b-4941-8996-1f264a5b08a0 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:10.759634 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8bd73b-4ef8-4d0b-b669-5b11cb848db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:10.760413 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8bd73b-4ef8-4d0b-b669-5b11cb848db5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:10.760696 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8bd73b-4ef8-4d0b-b669-5b11cb848db5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:10.760949 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8bd73b-4ef8-4d0b-b669-5b11cb848db5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:10.771669 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8bd73b-4ef8-4d0b-b669-5b11cb848db5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:10.773111 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8bd73b-4ef8-4d0b-b669-5b11cb848db5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:10.780573 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:10.791643 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-be8bd73b-4ef8-4d0b-b669-5b11cb848db5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:10.791643 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:10.791643 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:10.791643 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:10.791643 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:10.791643 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:10.791643 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:10.791643 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:10.791643 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:10.791643 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:10.791643 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:10.791643 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:10.791643 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:10.791643 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:10.791643 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:10.792678 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:10.792678 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:10.792678 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:10.792678 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:10.792678 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:10.792678 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:10.792678 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:10.792678 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:10.792678 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:10.792678 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:10.792678 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:10.792678 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:10.792678 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:10.792678 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:10.792678 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:10.792678 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:10.792678 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:10.793576 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:10.793576 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:10.793576 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:10.793576 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:10.793576 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 263/522] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:10.793576 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8bd73b-4ef8-4d0b-b669-5b11cb848db5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:10.794594 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90064a5e-f169-4419-a1c6-60c8ec222e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:10.795293 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90064a5e-f169-4419-a1c6-60c8ec222e0f None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:10.795580 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90064a5e-f169-4419-a1c6-60c8ec222e0f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:10.795878 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90064a5e-f169-4419-a1c6-60c8ec222e0f None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:10.796837 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90064a5e-f169-4419-a1c6-60c8ec222e0f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:10.798454 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90064a5e-f169-4419-a1c6-60c8ec222e0f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:10.817581 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:10.817933 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 260/523] 10.222.0.50 () {60 vars in 1132 bytes} [Fri May 27 15:01:10 2022] POST /identity/v3/users => generated 247 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:01:10.818734 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90064a5e-f169-4419-a1c6-60c8ec222e0f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:10.944832 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8bd73b-4ef8-4d0b-b669-5b11cb848db5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:10.957233 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90064a5e-f169-4419-a1c6-60c8ec222e0f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:11.007087 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7f3822-bc41-4fa2-a9b6-dff0fc202dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:11.008605 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f3822-bc41-4fa2-a9b6-dff0fc202dc0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:11.009402 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d6fc60-c52b-463c-aaa7-ea7a3d6aae46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:11.010328 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6fc60-c52b-463c-aaa7-ea7a3d6aae46 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:11.010608 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6fc60-c52b-463c-aaa7-ea7a3d6aae46 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:11.012048 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d6fc60-c52b-463c-aaa7-ea7a3d6aae46 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:11.012173 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f3822-bc41-4fa2-a9b6-dff0fc202dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:11.012173 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7f3822-bc41-4fa2-a9b6-dff0fc202dc0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:11.012320 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d6fc60-c52b-463c-aaa7-ea7a3d6aae46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:11.012320 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d6fc60-c52b-463c-aaa7-ea7a3d6aae46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:11.014256 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d6fc60-c52b-463c-aaa7-ea7a3d6aae46 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:11.021796 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7f3822-bc41-4fa2-a9b6-dff0fc202dc0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:11.023938 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7f3822-bc41-4fa2-a9b6-dff0fc202dc0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:11.032075 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 261/524] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:11.032949 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d6fc60-c52b-463c-aaa7-ea7a3d6aae46 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:11.040246 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-bd7f3822-bc41-4fa2-a9b6-dff0fc202dc0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:11.040246 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:11.040246 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:11.040246 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:11.040246 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:11.040246 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:11.040246 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:11.040246 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:11.040246 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:11.040246 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:11.040246 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:11.040246 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:11.040246 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 15:01:11.040246 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 15:01:11.040246 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 15:01:11.041101 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 15:01:11.041101 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:11.041101 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:11.041101 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:11.041101 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:11.041101 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:11.041101 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:11.041101 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:11.041101 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:11.041101 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:11.041101 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:11.041101 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:11.041101 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:11.041101 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:11.041101 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:11.041101 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:11.041101 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:11.041971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 15:01:11.041971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 15:01:11.041971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 15:01:11.041971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 15:01:11.041971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 15:01:11.041971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 15:01:11.041971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:11.041971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:11.041971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 264/525] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:01:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:11.042455 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7f3822-bc41-4fa2-a9b6-dff0fc202dc0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:11.094341 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:11.145966 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:11.170743 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7f3822-bc41-4fa2-a9b6-dff0fc202dc0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:11.173380 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d6fc60-c52b-463c-aaa7-ea7a3d6aae46 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:11.221240 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cda312-0474-4c11-bd47-93672ba17c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:11.221930 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cda312-0474-4c11-bd47-93672ba17c1f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:11.222225 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cda312-0474-4c11-bd47-93672ba17c1f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:11.222508 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cda312-0474-4c11-bd47-93672ba17c1f None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:11.223465 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cda312-0474-4c11-bd47-93672ba17c1f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:11.223571 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5f4187-9762-4236-9b15-ba4714b91d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:11.223785 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cda312-0474-4c11-bd47-93672ba17c1f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:11.224232 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f4187-9762-4236-9b15-ba4714b91d5e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:11.224516 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f4187-9762-4236-9b15-ba4714b91d5e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:11.224791 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5f4187-9762-4236-9b15-ba4714b91d5e None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:11.225227 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cda312-0474-4c11-bd47-93672ba17c1f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:11.225518 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5f4187-9762-4236-9b15-ba4714b91d5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:11.225822 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5f4187-9762-4236-9b15-ba4714b91d5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:11.227667 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5f4187-9762-4236-9b15-ba4714b91d5e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:11.248657 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 265/526] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:11.249222 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 262/527] 10.222.0.50 () {56 vars in 1139 bytes} [Fri May 27 15:01:11 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:11.251012 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cda312-0474-4c11-bd47-93672ba17c1f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:11.253651 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5f4187-9762-4236-9b15-ba4714b91d5e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:11.365729 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:11.369820 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5f4187-9762-4236-9b15-ba4714b91d5e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:11.373070 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cda312-0474-4c11-bd47-93672ba17c1f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:11.408794 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0edfc7-c9d8-4d88-af84-9ed6a620e893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:11.409705 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0edfc7-c9d8-4d88-af84-9ed6a620e893 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:11.409995 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0edfc7-c9d8-4d88-af84-9ed6a620e893 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:11.410209 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c213c4-b61a-463f-a9a2-c70aa9e35231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:11.410323 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0edfc7-c9d8-4d88-af84-9ed6a620e893 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:11.411018 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c213c4-b61a-463f-a9a2-c70aa9e35231 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:11.411119 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0edfc7-c9d8-4d88-af84-9ed6a620e893 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:11.411299 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c213c4-b61a-463f-a9a2-c70aa9e35231 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:11.411430 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0edfc7-c9d8-4d88-af84-9ed6a620e893 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:11.411622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c213c4-b61a-463f-a9a2-c70aa9e35231 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:11.412819 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0edfc7-c9d8-4d88-af84-9ed6a620e893 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:11.418338 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c213c4-b61a-463f-a9a2-c70aa9e35231 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:11.419791 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c213c4-b61a-463f-a9a2-c70aa9e35231 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:11.435880 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 266/528] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 15:01:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:11.435880 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0edfc7-c9d8-4d88-af84-9ed6a620e893 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:11.436113 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-f2c213c4-b61a-463f-a9a2-c70aa9e35231 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:11.436113 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:11.436113 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:11.436113 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:11.436113 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:11.436113 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:11.436113 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:11.436113 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:11.436113 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:11.436113 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:11.436113 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:11.436113 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:11.436113 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 15:01:11.436113 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 15:01:11.436113 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 15:01:11.437621 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 15:01:11.437621 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:11.437621 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:11.437621 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:11.437621 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:11.437621 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:11.437621 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:11.437621 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:11.437621 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:11.437621 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:11.437621 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:11.437621 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:11.437621 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:11.437621 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:11.437621 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:11.437621 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:11.437621 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:11.438659 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 15:01:11.438659 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 15:01:11.438659 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 15:01:11.438659 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 15:01:11.438659 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 15:01:11.438659 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 15:01:11.438659 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:11.438659 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:11.438659 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 263/529] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:01:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:11.438659 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c213c4-b61a-463f-a9a2-c70aa9e35231 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:11.445231 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:11.548586 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c213c4-b61a-463f-a9a2-c70aa9e35231 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:11.549154 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0edfc7-c9d8-4d88-af84-9ed6a620e893 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:11.585196 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75ad8a7-8214-4925-8856-27e33932b736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:11.586021 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ad8a7-8214-4925-8856-27e33932b736 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:11.586307 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ad8a7-8214-4925-8856-27e33932b736 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:11.586584 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75ad8a7-8214-4925-8856-27e33932b736 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:11.587212 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8600f80f-9f14-49c1-9e72-85b4d48c051c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:11.588155 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600f80f-9f14-49c1-9e72-85b4d48c051c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:11.588450 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600f80f-9f14-49c1-9e72-85b4d48c051c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:11.588737 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8600f80f-9f14-49c1-9e72-85b4d48c051c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:11.594010 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75ad8a7-8214-4925-8856-27e33932b736 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:11.595718 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75ad8a7-8214-4925-8856-27e33932b736 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:11.611101 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-f75ad8a7-8214-4925-8856-27e33932b736 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:11.611101 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:11.611101 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:11.611101 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:11.611101 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:11.611101 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:11.611101 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:11.611101 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:11.611101 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:11.611101 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:11.611101 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:11.611101 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:11.611101 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 15:01:11.611101 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 15:01:11.611101 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 15:01:11.612056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 15:01:11.612056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:11.612056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:11.612056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:11.612056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:11.612056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:11.612056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:11.612056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:11.612056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:11.612056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:11.612056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:11.612056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:11.612056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:11.612056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:11.612056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:11.612056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:11.612056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:11.613088 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 15:01:11.613088 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 15:01:11.613088 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 15:01:11.613088 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 15:01:11.613088 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 15:01:11.613088 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 15:01:11.613088 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:11.613088 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:11.614843 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 264/530] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:01:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:11.615909 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75ad8a7-8214-4925-8856-27e33932b736 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:11.634070 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:11.667439 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8600f80f-9f14-49c1-9e72-85b4d48c051c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:11.668955 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8600f80f-9f14-49c1-9e72-85b4d48c051c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:11.673362 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:11.725483 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75ad8a7-8214-4925-8856-27e33932b736 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:11.763853 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25af0c1c-0cbd-42ba-aa4a-f6f9c920359f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:11.764817 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af0c1c-0cbd-42ba-aa4a-f6f9c920359f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:11.765198 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af0c1c-0cbd-42ba-aa4a-f6f9c920359f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:11.765834 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af0c1c-0cbd-42ba-aa4a-f6f9c920359f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:11.766789 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25af0c1c-0cbd-42ba-aa4a-f6f9c920359f None admin] RBAC: Adding query filter params (role.id=13f37584013544e7ad1e2c8f350da6d2, scope.project.id=daf12585428b41a2b45bdbc85413a6c4, user.id=451a9232f95e494683d81b6078a318d0) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:11.768794 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25af0c1c-0cbd-42ba-aa4a-f6f9c920359f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:11.768883 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25af0c1c-0cbd-42ba-aa4a-f6f9c920359f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:11.786885 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 267/531] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:01:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 15:01:11.786885 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8600f80f-9f14-49c1-9e72-85b4d48c051c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:11.787197 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 265/532] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:01:11 2022] GET /identity/v3/role_assignments?user.id=451a9232f95e494683d81b6078a318d0&scope.project.id=daf12585428b41a2b45bdbc85413a6c4&role.id=13f37584013544e7ad1e2c8f350da6d2 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:11.787197 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25af0c1c-0cbd-42ba-aa4a-f6f9c920359f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:11.879849 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:11.903469 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8600f80f-9f14-49c1-9e72-85b4d48c051c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:11.908188 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:11.913692 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25af0c1c-0cbd-42ba-aa4a-f6f9c920359f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:11.945282 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f987402-6009-4de2-aa01-97803975f005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:11.946219 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f987402-6009-4de2-aa01-97803975f005 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:11.946685 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f987402-6009-4de2-aa01-97803975f005 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:11.947182 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f987402-6009-4de2-aa01-97803975f005 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:11.948135 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f987402-6009-4de2-aa01-97803975f005 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:11.948595 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f987402-6009-4de2-aa01-97803975f005 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:11.950210 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f987402-6009-4de2-aa01-97803975f005 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:11.963638 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3eaf89-3307-4c2f-9b0f-ea3a8a1534fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:11.964469 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3eaf89-3307-4c2f-9b0f-ea3a8a1534fd None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:11.964823 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3eaf89-3307-4c2f-9b0f-ea3a8a1534fd None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:11.965111 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3eaf89-3307-4c2f-9b0f-ea3a8a1534fd None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:11.965923 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3eaf89-3307-4c2f-9b0f-ea3a8a1534fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:11.966238 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3eaf89-3307-4c2f-9b0f-ea3a8a1534fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:11.966914 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 268/533] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 15:01:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:11.967745 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3eaf89-3307-4c2f-9b0f-ea3a8a1534fd None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:11.968815 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f987402-6009-4de2-aa01-97803975f005 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:11.993093 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 266/534] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 15:01:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:11.994412 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3eaf89-3307-4c2f-9b0f-ea3a8a1534fd None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:12.098420 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f987402-6009-4de2-aa01-97803975f005 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:12.120148 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3eaf89-3307-4c2f-9b0f-ea3a8a1534fd None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:12.134365 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de63f9ac-8ea8-4d70-b07c-04f7c7d58995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:12.135198 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de63f9ac-8ea8-4d70-b07c-04f7c7d58995 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:12.135543 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de63f9ac-8ea8-4d70-b07c-04f7c7d58995 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:12.135883 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de63f9ac-8ea8-4d70-b07c-04f7c7d58995 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:12.148222 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de63f9ac-8ea8-4d70-b07c-04f7c7d58995 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:12.151589 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de63f9ac-8ea8-4d70-b07c-04f7c7d58995 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:12.163011 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac72ac-cdc2-4c92-b6e2-3278c4bcec9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:12.163841 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac72ac-cdc2-4c92-b6e2-3278c4bcec9b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:12.164285 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac72ac-cdc2-4c92-b6e2-3278c4bcec9b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:12.164712 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeac72ac-cdc2-4c92-b6e2-3278c4bcec9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:12.165684 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeac72ac-cdc2-4c92-b6e2-3278c4bcec9b None admin] RBAC: Adding query filter params (role.id=13f37584013544e7ad1e2c8f350da6d2, scope.project.id=daf12585428b41a2b45bdbc85413a6c4, user.id=1afda9d6b1d64e6ab400b26124cf1290) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:12.166156 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeac72ac-cdc2-4c92-b6e2-3278c4bcec9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:12.167960 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeac72ac-cdc2-4c92-b6e2-3278c4bcec9b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:12.172852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-de63f9ac-8ea8-4d70-b07c-04f7c7d58995 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:12.172852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:12.172852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:12.172852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:12.172852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:12.172852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:12.172852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:12.172852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:12.172852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:12.172852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:12.172852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:12.172852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:12.172852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:12.172852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:12.172852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:12.173869 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:12.173869 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:12.173869 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:12.173869 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:12.173869 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:12.173869 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:12.173869 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:12.173869 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:12.173869 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:12.173869 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:12.173869 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:12.173869 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:12.173869 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:12.173869 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:12.173869 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:12.173869 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:12.173869 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:12.174999 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:12.174999 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:12.174999 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:12.174999 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:12.176749 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 269/535] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:12.177334 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 267/536] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:01:11 2022] GET /identity/v3/role_assignments?user.id=1afda9d6b1d64e6ab400b26124cf1290&scope.project.id=daf12585428b41a2b45bdbc85413a6c4&role.id=13f37584013544e7ad1e2c8f350da6d2 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:12.177733 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de63f9ac-8ea8-4d70-b07c-04f7c7d58995 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:12.183009 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac72ac-cdc2-4c92-b6e2-3278c4bcec9b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:12.188812 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:12.202033 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:12.308983 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de63f9ac-8ea8-4d70-b07c-04f7c7d58995 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:12.351350 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e017adab-9389-47d6-ac25-6b7e7b3c6c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:12.352419 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017adab-9389-47d6-ac25-6b7e7b3c6c38 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:12.354068 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017adab-9389-47d6-ac25-6b7e7b3c6c38 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:12.355399 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e017adab-9389-47d6-ac25-6b7e7b3c6c38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:12.356512 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e017adab-9389-47d6-ac25-6b7e7b3c6c38 None admin] RBAC: Adding query filter params (role.id=13f37584013544e7ad1e2c8f350da6d2, scope.project.id=daf12585428b41a2b45bdbc85413a6c4, user.id=896af6ccf89847e5903f6a4fe3d9d810) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:12.358647 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e017adab-9389-47d6-ac25-6b7e7b3c6c38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:12.360944 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e017adab-9389-47d6-ac25-6b7e7b3c6c38 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:12.375999 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 270/537] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:01:12 2022] GET /identity/v3/role_assignments?user.id=896af6ccf89847e5903f6a4fe3d9d810&scope.project.id=daf12585428b41a2b45bdbc85413a6c4&role.id=13f37584013544e7ad1e2c8f350da6d2 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:12.413364 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeac72ac-cdc2-4c92-b6e2-3278c4bcec9b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:12.462028 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ce1ebc-84d7-4612-8b7e-e4172bfad7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:12.463345 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce1ebc-84d7-4612-8b7e-e4172bfad7a9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:12.463836 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce1ebc-84d7-4612-8b7e-e4172bfad7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:12.464311 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ce1ebc-84d7-4612-8b7e-e4172bfad7a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:12.465275 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ce1ebc-84d7-4612-8b7e-e4172bfad7a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:12.465815 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ce1ebc-84d7-4612-8b7e-e4172bfad7a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:12.467489 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ce1ebc-84d7-4612-8b7e-e4172bfad7a9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:12.480840 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 268/538] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:12.488007 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e017adab-9389-47d6-ac25-6b7e7b3c6c38 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:12.595330 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:12.621877 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e017adab-9389-47d6-ac25-6b7e7b3c6c38 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:12.673259 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41e541f-d776-46be-a673-5f8444dad1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:12.673996 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41e541f-d776-46be-a673-5f8444dad1ad None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:12.674279 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41e541f-d776-46be-a673-5f8444dad1ad None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:12.674556 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41e541f-d776-46be-a673-5f8444dad1ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:12.682246 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41e541f-d776-46be-a673-5f8444dad1ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:12.683731 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41e541f-d776-46be-a673-5f8444dad1ad None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:12.701430 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-d41e541f-d776-46be-a673-5f8444dad1ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:12.701430 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:12.701430 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:12.701430 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:12.701430 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:12.701430 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:12.701430 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:12.701430 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:12.701430 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:12.701430 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:12.701430 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:12.701430 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:12.701430 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 15:01:12.701430 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 15:01:12.701430 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 15:01:12.702348 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 15:01:12.702348 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:12.702348 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:12.702348 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:12.702348 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:12.702348 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:12.702348 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:12.702348 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:12.702348 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:12.702348 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:12.702348 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:12.702348 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:12.702348 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:12.702348 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:12.702348 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:12.702348 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:12.702348 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:12.703301 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 15:01:12.703301 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 15:01:12.703301 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 15:01:12.703301 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 15:01:12.703301 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 15:01:12.703301 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 15:01:12.703301 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:12.703301 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:12.703721 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 271/539] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:01:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:12.709969 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ce1ebc-84d7-4612-8b7e-e4172bfad7a9 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:12.829765 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:12.882993 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ce1ebc-84d7-4612-8b7e-e4172bfad7a9 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:12.935658 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d984a06-5ec8-46a3-8fdb-a083da730479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:12.936663 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d984a06-5ec8-46a3-8fdb-a083da730479 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:12.937216 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d984a06-5ec8-46a3-8fdb-a083da730479 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:12.937717 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d984a06-5ec8-46a3-8fdb-a083da730479 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:12.938725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d984a06-5ec8-46a3-8fdb-a083da730479 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:12.939262 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d984a06-5ec8-46a3-8fdb-a083da730479 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:12.940901 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d984a06-5ec8-46a3-8fdb-a083da730479 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:12.955028 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 269/540] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 15:01:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:12.964170 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41e541f-d776-46be-a673-5f8444dad1ad None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:13.100915 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41e541f-d776-46be-a673-5f8444dad1ad None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:13.146344 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28724157-39c8-4c6b-86f3-314005994d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:13.147480 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28724157-39c8-4c6b-86f3-314005994d3f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:13.147761 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28724157-39c8-4c6b-86f3-314005994d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:13.148035 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28724157-39c8-4c6b-86f3-314005994d3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:13.148861 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28724157-39c8-4c6b-86f3-314005994d3f None admin] RBAC: Adding query filter params (role.id=13f37584013544e7ad1e2c8f350da6d2, scope.project.id=daf12585428b41a2b45bdbc85413a6c4, user.id=f874edc08438434189f1d14d79291487) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:13.149165 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28724157-39c8-4c6b-86f3-314005994d3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:13.150592 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28724157-39c8-4c6b-86f3-314005994d3f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:13.159970 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 272/541] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:01:12 2022] GET /identity/v3/role_assignments?user.id=f874edc08438434189f1d14d79291487&scope.project.id=daf12585428b41a2b45bdbc85413a6c4&role.id=13f37584013544e7ad1e2c8f350da6d2 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:13.282199 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e179746c-6484-40dd-b24c-c5a75107d644 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:13.282901 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e179746c-6484-40dd-b24c-c5a75107d644 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:13.283392 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e179746c-6484-40dd-b24c-c5a75107d644 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:13.285425 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 270/542] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:13.290952 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c35bc15-a6a7-4d81-a54a-9969ec057b59 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:13.291247 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c35bc15-a6a7-4d81-a54a-9969ec057b59 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:13.291525 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c35bc15-a6a7-4d81-a54a-9969ec057b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:13.301908 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:13.317902 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5c35bc15-a6a7-4d81-a54a-9969ec057b59 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:13.319105 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5c35bc15-a6a7-4d81-a54a-9969ec057b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:13.406838 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 273/543] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:01:13.443329 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e179746c-6484-40dd-b24c-c5a75107d644 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:13.587287 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e179746c-6484-40dd-b24c-c5a75107d644 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:13.616951 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:13.638520 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eca388-5121-4098-8b55-1a8db6d6505b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:13.639357 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eca388-5121-4098-8b55-1a8db6d6505b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:13.639640 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eca388-5121-4098-8b55-1a8db6d6505b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:13.639916 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eca388-5121-4098-8b55-1a8db6d6505b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:13.656453 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eca388-5121-4098-8b55-1a8db6d6505b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:01:13.664208 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-f6eca388-5121-4098-8b55-1a8db6d6505b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 15:01:13.664208 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:13.664208 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:13.664208 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:13.664208 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:13.664208 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:13.664208 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:13.664208 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:13.664208 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:13.664208 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:13.664208 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:13.664208 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:13.664208 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:01:13.664208 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:01:13.664208 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:01:13.664208 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:01:13.665098 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:01:13.665098 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:01:13.665098 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:01:13.665098 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:01:13.665098 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:13.665098 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:13.665098 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:13.665098 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:13.665098 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:13.665098 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:13.665098 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:13.665098 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:13.665098 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:13.665098 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:13.665098 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:13.665098 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:13.665906 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:13.665906 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:13.665906 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:13.665906 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:13.665906 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:01:13.665906 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:01:13.665906 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:01:13.665906 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:01:13.665906 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:01:13.665906 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:01:13.665906 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 15:01:13.665906 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:13.665906 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 271/544] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 15:01:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:13.672315 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c35bc15-a6a7-4d81-a54a-9969ec057b59 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:13.775405 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c35bc15-a6a7-4d81-a54a-9969ec057b59 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:13.811840 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7538e8-cd19-4331-9e83-7bf0a0f79930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:13.812868 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7538e8-cd19-4331-9e83-7bf0a0f79930 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:13.813149 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7538e8-cd19-4331-9e83-7bf0a0f79930 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:13.813426 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7538e8-cd19-4331-9e83-7bf0a0f79930 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:13.814176 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7538e8-cd19-4331-9e83-7bf0a0f79930 None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:13.814478 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7538e8-cd19-4331-9e83-7bf0a0f79930 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:13.815894 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7538e8-cd19-4331-9e83-7bf0a0f79930 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:13.817377 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:13.831147 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eca388-5121-4098-8b55-1a8db6d6505b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:13.841723 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 274/545] 10.222.0.50 () {56 vars in 1103 bytes} [Fri May 27 15:01:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:13.943523 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eca388-5121-4098-8b55-1a8db6d6505b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:13.990240 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03eb8d3-8c93-4c5f-9804-32c9b7449b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:13.991079 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03eb8d3-8c93-4c5f-9804-32c9b7449b40 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:13.991363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03eb8d3-8c93-4c5f-9804-32c9b7449b40 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:13.991637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03eb8d3-8c93-4c5f-9804-32c9b7449b40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:14.011182 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03eb8d3-8c93-4c5f-9804-32c9b7449b40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:14.012663 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03eb8d3-8c93-4c5f-9804-32c9b7449b40 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:14.034194 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-b03eb8d3-8c93-4c5f-9804-32c9b7449b40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:14.034194 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:14.034194 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:14.034194 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:14.034194 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:14.034194 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:14.034194 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:14.034194 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:14.034194 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:14.034194 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:14.034194 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:14.034194 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:14.034194 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:14.034194 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:14.034194 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:14.035112 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:14.035112 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:14.035112 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:14.035112 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:14.035112 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:14.035112 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:14.035112 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:14.035112 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:14.035112 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:14.035112 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:14.035112 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:14.035112 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:14.035112 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:14.035112 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:14.035112 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:14.035112 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:14.035112 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:14.035965 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:14.035965 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:14.035965 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:14.035965 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:14.035965 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 272/546] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:01:14.040917 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7538e8-cd19-4331-9e83-7bf0a0f79930 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:14.187944 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7538e8-cd19-4331-9e83-7bf0a0f79930 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:14.232694 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57648c0-a9c0-4728-9286-56ab850edc3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:14.233365 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57648c0-a9c0-4728-9286-56ab850edc3a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:14.233648 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57648c0-a9c0-4728-9286-56ab850edc3a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:14.233993 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57648c0-a9c0-4728-9286-56ab850edc3a None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:14.234723 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57648c0-a9c0-4728-9286-56ab850edc3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:14.235054 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57648c0-a9c0-4728-9286-56ab850edc3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:14.236421 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57648c0-a9c0-4728-9286-56ab850edc3a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:14.255920 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 275/547] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:14.261140 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03eb8d3-8c93-4c5f-9804-32c9b7449b40 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:14.308181 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:14.404357 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03eb8d3-8c93-4c5f-9804-32c9b7449b40 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:14.454452 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cc35d7-e27f-4c82-8a99-ef77c3c4694a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:14.456108 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cc35d7-e27f-4c82-8a99-ef77c3c4694a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:14.456598 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cc35d7-e27f-4c82-8a99-ef77c3c4694a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:14.457112 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cc35d7-e27f-4c82-8a99-ef77c3c4694a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:14.471268 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cc35d7-e27f-4c82-8a99-ef77c3c4694a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:14.473055 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cc35d7-e27f-4c82-8a99-ef77c3c4694a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:14.494580 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-84cc35d7-e27f-4c82-8a99-ef77c3c4694a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 15:01:14.494580 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:14.494580 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:14.494580 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:14.494580 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:14.494580 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:14.494580 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:14.494580 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:14.494580 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:14.494580 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:14.494580 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:14.494580 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:14.494580 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 15:01:14.494580 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 15:01:14.494580 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 15:01:14.494580 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 15:01:14.496072 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:14.496072 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:14.496072 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 15:01:14.496072 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:14.496072 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 15:01:14.496072 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:14.496072 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:14.496072 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:14.496072 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:14.496072 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:14.496072 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:14.496072 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:14.496072 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:14.496072 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:14.496072 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:14.496072 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:14.496072 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:14.496072 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:14.497147 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:14.497147 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:14.497147 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 15:01:14.497147 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 15:01:14.497147 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 15:01:14.497147 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 15:01:14.497147 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 15:01:14.497147 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 15:01:14.497147 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 15:01:14.497147 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:14.500077 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 273/548] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 15:01:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:14.505548 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57648c0-a9c0-4728-9286-56ab850edc3a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:14.589131 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:14.650523 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57648c0-a9c0-4728-9286-56ab850edc3a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:14.704176 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5d9579-59a8-466d-88c8-1eca346847e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:14.704774 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d9579-59a8-466d-88c8-1eca346847e1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:14.705062 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d9579-59a8-466d-88c8-1eca346847e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:14.705376 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5d9579-59a8-466d-88c8-1eca346847e1 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:14.706225 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5d9579-59a8-466d-88c8-1eca346847e1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:14.706530 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5d9579-59a8-466d-88c8-1eca346847e1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:14.708004 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5d9579-59a8-466d-88c8-1eca346847e1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:14.732410 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 276/549] 10.222.0.50 () {56 vars in 1135 bytes} [Fri May 27 15:01:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:14.738032 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cc35d7-e27f-4c82-8a99-ef77c3c4694a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:14.879833 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903a2cb-d5b8-47c3-b8d6-91098d2c2f63 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:14.880499 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903a2cb-d5b8-47c3-b8d6-91098d2c2f63 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:14.881003 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903a2cb-d5b8-47c3-b8d6-91098d2c2f63 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:14.886963 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 277/550] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 15:01:14.888741 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8952694-024e-42f9-a277-e66e4b23cdcb None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:14.889252 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8952694-024e-42f9-a277-e66e4b23cdcb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:14.889744 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8952694-024e-42f9-a277-e66e4b23cdcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:14.902009 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:14.918830 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cc35d7-e27f-4c82-8a99-ef77c3c4694a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:14.931131 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b8952694-024e-42f9-a277-e66e4b23cdcb None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:14.932474 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b8952694-024e-42f9-a277-e66e4b23cdcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:14.989437 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcaf4f8-850c-43d7-82bb-86b06cedba9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:14.990007 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcaf4f8-850c-43d7-82bb-86b06cedba9f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:14.990288 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcaf4f8-850c-43d7-82bb-86b06cedba9f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:14.990567 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcaf4f8-850c-43d7-82bb-86b06cedba9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:15.045626 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 278/551] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 15:01:15.048482 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e17f25-052e-46cb-9a22-851e6f01c258 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:15.048773 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e17f25-052e-46cb-9a22-851e6f01c258 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:15.049051 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e17f25-052e-46cb-9a22-851e6f01c258 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:15.051288 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 279/552] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:15.056336 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818318d6-c708-4d34-8b2d-9ea6c2bbca1e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:15.056725 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818318d6-c708-4d34-8b2d-9ea6c2bbca1e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:15.057258 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818318d6-c708-4d34-8b2d-9ea6c2bbca1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:15.078942 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-818318d6-c708-4d34-8b2d-9ea6c2bbca1e None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:15.080462 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-818318d6-c708-4d34-8b2d-9ea6c2bbca1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:15.090343 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcaf4f8-850c-43d7-82bb-86b06cedba9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:15.091878 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcaf4f8-850c-43d7-82bb-86b06cedba9f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:15.133940 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:15.178761 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 280/553] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:01:15.180419 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818318d6-c708-4d34-8b2d-9ea6c2bbca1e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:15.198886 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:15.206622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 274/554] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:01:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 15:01:15.206622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7e006-9c2b-462b-b64e-5ddf043380c2 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:15.206622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7e006-9c2b-462b-b64e-5ddf043380c2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:15.206622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf7e006-9c2b-462b-b64e-5ddf043380c2 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:15.207768 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 275/555] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:15.208302 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf7e006-9c2b-462b-b64e-5ddf043380c2 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:15.301123 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818318d6-c708-4d34-8b2d-9ea6c2bbca1e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:15.311220 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf7e006-9c2b-462b-b64e-5ddf043380c2 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:15.337796 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a0201e-5344-4808-8296-9f55d126b66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:15.339194 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0201e-5344-4808-8296-9f55d126b66e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:15.339661 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0201e-5344-4808-8296-9f55d126b66e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:15.340113 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0201e-5344-4808-8296-9f55d126b66e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:15.348184 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ed51c-f0d6-447d-8fd1-0f706b69ad15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:15.348970 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ed51c-f0d6-447d-8fd1-0f706b69ad15 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:15.349249 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ed51c-f0d6-447d-8fd1-0f706b69ad15 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:15.349519 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245ed51c-f0d6-447d-8fd1-0f706b69ad15 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:15.352390 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a0201e-5344-4808-8296-9f55d126b66e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:01:15.359710 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245ed51c-f0d6-447d-8fd1-0f706b69ad15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:01:15.360829 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-12a0201e-5344-4808-8296-9f55d126b66e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 15:01:15.360829 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:15.360829 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:15.360829 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:15.360829 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:15.360829 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:15.360829 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:15.360829 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:15.360829 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:15.360829 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:15.360829 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:15.360829 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:15.360829 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:01:15.360829 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:01:15.360829 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:01:15.360829 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:01:15.361720 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:01:15.361720 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:01:15.361720 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:01:15.361720 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:01:15.361720 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:15.361720 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:15.361720 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:15.361720 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:15.361720 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:15.361720 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:15.361720 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:15.361720 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:15.361720 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:15.361720 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:15.361720 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:15.361720 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:15.362545 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:15.362545 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:15.362545 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:15.362545 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:15.362545 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:01:15.362545 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:01:15.362545 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:01:15.362545 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:01:15.362545 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:01:15.362545 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:01:15.362545 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 15:01:15.362545 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:15.364474 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 281/556] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 15:01:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:15.365791 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603c49e-fc45-48b5-bcbc-76f115c8ce94 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:15.366288 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603c49e-fc45-48b5-bcbc-76f115c8ce94 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:15.366708 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8603c49e-fc45-48b5-bcbc-76f115c8ce94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:15.367437 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-245ed51c-f0d6-447d-8fd1-0f706b69ad15 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 15:01:15.367437 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:15.367437 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:15.367437 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:15.367437 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:15.367437 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:15.367437 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:15.367437 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:15.367437 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:15.367437 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:15.367437 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:15.367437 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:15.367437 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:01:15.367437 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:01:15.367437 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:01:15.367437 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:01:15.368311 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:01:15.368311 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:01:15.368311 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:01:15.368311 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:01:15.368311 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:15.368311 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:15.368311 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:15.368311 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:15.368311 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:15.368311 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:15.368311 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:15.368311 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:15.368311 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:15.368311 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:15.368311 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:15.368311 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:15.369116 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:15.369116 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:15.369116 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:15.369116 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:15.369116 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:01:15.369116 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:01:15.369116 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:01:15.369116 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:01:15.369116 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:01:15.369116 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:01:15.369116 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 15:01:15.369116 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:15.369116 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 276/557] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 15:01:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:15.369796 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ed51c-f0d6-447d-8fd1-0f706b69ad15 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:15.395177 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8603c49e-fc45-48b5-bcbc-76f115c8ce94 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:15.397361 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8603c49e-fc45-48b5-bcbc-76f115c8ce94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:15.432790 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:15.437079 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:15.478774 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 282/558] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:01:15.479645 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8603c49e-fc45-48b5-bcbc-76f115c8ce94 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:15.498666 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245ed51c-f0d6-447d-8fd1-0f706b69ad15 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:15.539972 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a54478-a6c7-450a-bca1-de40622ce7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:15.540772 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a54478-a6c7-450a-bca1-de40622ce7dc None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:15.541083 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a54478-a6c7-450a-bca1-de40622ce7dc None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:15.541355 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a54478-a6c7-450a-bca1-de40622ce7dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:15.553037 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a54478-a6c7-450a-bca1-de40622ce7dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:15.554526 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a54478-a6c7-450a-bca1-de40622ce7dc None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:15.573735 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-e8a54478-a6c7-450a-bca1-de40622ce7dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:15.573735 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:15.573735 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:15.573735 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:15.573735 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:15.573735 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:15.573735 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:15.573735 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:15.573735 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:15.573735 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:15.573735 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:15.573735 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:15.573735 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:15.573735 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:15.573735 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:15.575180 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:15.575180 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:15.575180 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:15.575180 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:15.575180 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:15.575180 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:15.575180 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:15.575180 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:15.575180 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:15.575180 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:15.575180 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:15.575180 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:15.575180 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:15.575180 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:15.575180 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:15.575180 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:15.575180 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:15.576059 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:15.576059 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:15.576059 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:15.576059 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:15.576059 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 277/559] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:15.576059 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a54478-a6c7-450a-bca1-de40622ce7dc None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:15.592404 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8603c49e-fc45-48b5-bcbc-76f115c8ce94 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:15.632904 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4007b98-ff1e-48e6-ba6b-ebe709a2f5a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:15.633726 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4007b98-ff1e-48e6-ba6b-ebe709a2f5a5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:15.634006 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4007b98-ff1e-48e6-ba6b-ebe709a2f5a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:15.634277 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4007b98-ff1e-48e6-ba6b-ebe709a2f5a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:15.635046 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4007b98-ff1e-48e6-ba6b-ebe709a2f5a5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:15.635595 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4007b98-ff1e-48e6-ba6b-ebe709a2f5a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:15.636716 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4007b98-ff1e-48e6-ba6b-ebe709a2f5a5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:15.651311 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 283/560] 10.222.0.50 () {56 vars in 1103 bytes} [Fri May 27 15:01:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:15.651311 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4007b98-ff1e-48e6-ba6b-ebe709a2f5a5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:15.681953 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:15.690155 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a54478-a6c7-450a-bca1-de40622ce7dc None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:15.700384 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:15.727914 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca7f1a0-1c08-4971-9bcb-14ca96ceafeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:15.728584 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7f1a0-1c08-4971-9bcb-14ca96ceafeb None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:15.728866 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7f1a0-1c08-4971-9bcb-14ca96ceafeb None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:15.729139 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca7f1a0-1c08-4971-9bcb-14ca96ceafeb None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:15.729877 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca7f1a0-1c08-4971-9bcb-14ca96ceafeb None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:15.730184 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca7f1a0-1c08-4971-9bcb-14ca96ceafeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:15.731586 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca7f1a0-1c08-4971-9bcb-14ca96ceafeb None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:15.741311 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 278/561] 10.222.0.50 () {56 vars in 1103 bytes} [Fri May 27 15:01:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:15.742741 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca7f1a0-1c08-4971-9bcb-14ca96ceafeb None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:15.763803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4007b98-ff1e-48e6-ba6b-ebe709a2f5a5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:15.802284 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe07db0-4acd-4201-97d7-0be2b52de539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:15.803197 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe07db0-4acd-4201-97d7-0be2b52de539 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:15.803377 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe07db0-4acd-4201-97d7-0be2b52de539 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:15.803648 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe07db0-4acd-4201-97d7-0be2b52de539 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:15.815308 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe07db0-4acd-4201-97d7-0be2b52de539 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:01:15.822887 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-bfe07db0-4acd-4201-97d7-0be2b52de539 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 15:01:15.822887 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:15.822887 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:15.822887 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:15.822887 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:15.822887 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:15.822887 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:15.822887 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:15.822887 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:15.822887 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:15.822887 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:15.822887 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:15.822887 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:01:15.822887 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:01:15.822887 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:01:15.822887 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:01:15.823852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:01:15.823852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:01:15.823852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:01:15.823852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:01:15.823852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:15.823852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:15.823852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:15.823852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:15.823852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:15.823852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:15.823852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:15.823852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:15.823852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:15.823852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:15.823852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:15.823852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:15.830654 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:15.830654 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:15.830654 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:15.830654 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:15.830654 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:01:15.830654 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:01:15.830654 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:01:15.830654 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:01:15.830654 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:01:15.830654 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:01:15.830654 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 15:01:15.830654 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:15.830654 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 284/562] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 15:01:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:15.830654 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe07db0-4acd-4201-97d7-0be2b52de539 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:15.856733 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca7f1a0-1c08-4971-9bcb-14ca96ceafeb None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:15.890684 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da9c8a-457b-4cb4-8fcf-06ea9324d2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:15.891596 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da9c8a-457b-4cb4-8fcf-06ea9324d2f6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:15.891872 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da9c8a-457b-4cb4-8fcf-06ea9324d2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:15.892146 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da9c8a-457b-4cb4-8fcf-06ea9324d2f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:15.892853 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27da9c8a-457b-4cb4-8fcf-06ea9324d2f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:15.893153 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27da9c8a-457b-4cb4-8fcf-06ea9324d2f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:15.894513 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27da9c8a-457b-4cb4-8fcf-06ea9324d2f6 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:15.905984 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 279/563] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:15.906926 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da9c8a-457b-4cb4-8fcf-06ea9324d2f6 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:15.916671 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:15.932836 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe07db0-4acd-4201-97d7-0be2b52de539 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:15.945553 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:15.970644 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abdba1a-d79f-4dc6-8ccd-d3a902c66e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:15.971480 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abdba1a-d79f-4dc6-8ccd-d3a902c66e2c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:15.971766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abdba1a-d79f-4dc6-8ccd-d3a902c66e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:15.972041 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abdba1a-d79f-4dc6-8ccd-d3a902c66e2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:15.983037 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abdba1a-d79f-4dc6-8ccd-d3a902c66e2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:15.985089 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abdba1a-d79f-4dc6-8ccd-d3a902c66e2c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:16.004462 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-3abdba1a-d79f-4dc6-8ccd-d3a902c66e2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:16.004462 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:16.004462 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:16.004462 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:16.004462 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:16.004462 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:16.004462 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:16.004462 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:16.004462 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:16.004462 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:16.004462 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:16.004462 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:16.004462 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:16.004462 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:16.004462 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:16.005399 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:16.005399 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:16.005399 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:16.005399 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:16.005399 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:16.005399 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:16.005399 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:16.005399 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:16.005399 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:16.005399 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:16.005399 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:16.005399 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:16.005399 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:16.005399 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:16.005399 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:16.005399 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:16.005399 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:16.006303 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:16.006303 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:16.006303 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:16.006303 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:16.006303 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 285/564] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:16.006589 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abdba1a-d79f-4dc6-8ccd-d3a902c66e2c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:16.018454 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da9c8a-457b-4cb4-8fcf-06ea9324d2f6 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:16.052338 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8705c966-d56e-4c65-b579-e40cde9af8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:16.053130 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705c966-d56e-4c65-b579-e40cde9af8f0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:16.053481 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705c966-d56e-4c65-b579-e40cde9af8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:16.053689 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705c966-d56e-4c65-b579-e40cde9af8f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:16.064283 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8705c966-d56e-4c65-b579-e40cde9af8f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:16.065756 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8705c966-d56e-4c65-b579-e40cde9af8f0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:16.082869 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-8705c966-d56e-4c65-b579-e40cde9af8f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:16.082869 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:16.082869 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:16.082869 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:16.082869 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:16.082869 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:16.082869 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:16.082869 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:16.082869 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:16.082869 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:16.082869 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:16.082869 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:16.082869 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:16.082869 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:16.082869 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:16.083940 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:16.083940 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:16.083940 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:16.083940 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:16.083940 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:16.083940 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:16.083940 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:16.083940 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:16.083940 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:16.083940 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:16.083940 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:16.083940 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:16.083940 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:16.083940 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:16.083940 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:16.083940 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:16.083940 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:16.084866 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:16.084866 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:16.084866 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:16.084866 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:16.084866 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 280/565] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:16.085359 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71555c-c840-4ebd-88bc-66234c90e054 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:16.085606 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71555c-c840-4ebd-88bc-66234c90e054 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:16.085880 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71555c-c840-4ebd-88bc-66234c90e054 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:16.087323 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 281/566] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:16.087921 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e71555c-c840-4ebd-88bc-66234c90e054 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:16.113287 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abdba1a-d79f-4dc6-8ccd-d3a902c66e2c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:16.146603 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd416f-47f5-4676-b390-73b46e6a125a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:16.147301 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd416f-47f5-4676-b390-73b46e6a125a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:16.147587 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd416f-47f5-4676-b390-73b46e6a125a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:16.147867 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd416f-47f5-4676-b390-73b46e6a125a None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:16.148608 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edd416f-47f5-4676-b390-73b46e6a125a None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:16.148917 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edd416f-47f5-4676-b390-73b46e6a125a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:16.150297 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edd416f-47f5-4676-b390-73b46e6a125a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:16.155449 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:16.165655 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 286/567] 10.222.0.50 () {56 vars in 1103 bytes} [Fri May 27 15:01:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:16.165655 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd416f-47f5-4676-b390-73b46e6a125a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:16.178477 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:16.198800 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e71555c-c840-4ebd-88bc-66234c90e054 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:16.231645 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0f326d-c53e-4b2d-ab84-9c675ad69a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:16.232426 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f326d-c53e-4b2d-ab84-9c675ad69a30 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:16.232706 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f326d-c53e-4b2d-ab84-9c675ad69a30 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:16.232979 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f326d-c53e-4b2d-ab84-9c675ad69a30 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:16.240115 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0f326d-c53e-4b2d-ab84-9c675ad69a30 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:16.241557 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0f326d-c53e-4b2d-ab84-9c675ad69a30 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:16.255479 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-9d0f326d-c53e-4b2d-ab84-9c675ad69a30 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:16.255479 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:16.255479 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:16.255479 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:16.255479 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:16.255479 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:16.255479 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:16.255479 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:16.255479 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:16.255479 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:16.255479 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:16.255479 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:16.255479 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 15:01:16.255479 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 15:01:16.255479 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 15:01:16.256357 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 15:01:16.256357 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:16.256357 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:16.256357 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:16.256357 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:16.256357 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:16.256357 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:16.256357 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:16.256357 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:16.256357 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:16.256357 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:16.256357 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:16.256357 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:16.256357 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:16.256357 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:16.256357 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:16.256357 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:16.257263 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 15:01:16.257263 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 15:01:16.257263 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 15:01:16.257263 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 15:01:16.257263 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 15:01:16.257263 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 15:01:16.257263 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:16.257263 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:16.257263 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 282/568] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:01:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:16.258163 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0f326d-c53e-4b2d-ab84-9c675ad69a30 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:16.262905 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd416f-47f5-4676-b390-73b46e6a125a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:16.297689 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b39d3db-b7e0-4008-b9c3-64f6b35e22d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:16.298358 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39d3db-b7e0-4008-b9c3-64f6b35e22d6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:16.298642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39d3db-b7e0-4008-b9c3-64f6b35e22d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:16.298953 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39d3db-b7e0-4008-b9c3-64f6b35e22d6 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:16.299673 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b39d3db-b7e0-4008-b9c3-64f6b35e22d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:16.299979 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b39d3db-b7e0-4008-b9c3-64f6b35e22d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:16.301337 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b39d3db-b7e0-4008-b9c3-64f6b35e22d6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:16.313237 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 287/569] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:16.314461 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c10da-8369-4124-b014-160e42fc6884 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:16.314877 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c10da-8369-4124-b014-160e42fc6884 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:16.315153 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c10da-8369-4124-b014-160e42fc6884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:16.331070 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-641c10da-8369-4124-b014-160e42fc6884 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:16.332258 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-641c10da-8369-4124-b014-160e42fc6884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:16.362327 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0f326d-c53e-4b2d-ab84-9c675ad69a30 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:16.387171 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:16.394920 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bca0492-a613-42b2-8227-678a796edcab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:16.395575 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bca0492-a613-42b2-8227-678a796edcab None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:16.395861 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bca0492-a613-42b2-8227-678a796edcab None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:16.396137 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bca0492-a613-42b2-8227-678a796edcab None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:16.396851 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bca0492-a613-42b2-8227-678a796edcab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:16.397159 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bca0492-a613-42b2-8227-678a796edcab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:16.398540 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bca0492-a613-42b2-8227-678a796edcab None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:16.399246 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 288/570] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:01:16.399993 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641c10da-8369-4124-b014-160e42fc6884 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:16.410831 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 283/571] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:16.410831 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bca0492-a613-42b2-8227-678a796edcab None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:16.423568 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:16.513183 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641c10da-8369-4124-b014-160e42fc6884 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:16.523554 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bca0492-a613-42b2-8227-678a796edcab None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:16.547195 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d0c869-54f3-4a8e-837c-74979cc1f69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:16.547987 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d0c869-54f3-4a8e-837c-74979cc1f69f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:16.548265 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d0c869-54f3-4a8e-837c-74979cc1f69f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:16.548538 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d0c869-54f3-4a8e-837c-74979cc1f69f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:16.557248 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b82e8-6198-4575-9cc6-847324a16ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:16.557904 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b82e8-6198-4575-9cc6-847324a16ef4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:16.558186 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b82e8-6198-4575-9cc6-847324a16ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:16.558383 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d0c869-54f3-4a8e-837c-74979cc1f69f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:16.558474 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0b82e8-6198-4575-9cc6-847324a16ef4 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:16.559249 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b82e8-6198-4575-9cc6-847324a16ef4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:16.559559 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b82e8-6198-4575-9cc6-847324a16ef4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:16.559846 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d0c869-54f3-4a8e-837c-74979cc1f69f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:16.560964 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0b82e8-6198-4575-9cc6-847324a16ef4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:16.572555 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 284/572] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 15:01:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:16.573289 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b82e8-6198-4575-9cc6-847324a16ef4 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:16.577195 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-87d0c869-54f3-4a8e-837c-74979cc1f69f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:16.577195 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:16.577195 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:16.577195 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:16.577195 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:16.577195 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:16.577195 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:16.577195 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:16.577195 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:16.577195 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:16.577195 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:16.577195 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:16.577195 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:16.577195 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:16.577195 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:16.578292 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:16.578292 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:16.578292 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:16.578292 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:16.578292 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:16.578292 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:16.578292 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:16.578292 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:16.578292 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:16.578292 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:16.578292 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:16.578292 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:16.578292 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:16.578292 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:16.578292 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:16.578292 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:16.578292 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:16.579333 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:16.579333 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:16.579333 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:16.579333 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:16.579333 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 289/573] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:16.579333 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d0c869-54f3-4a8e-837c-74979cc1f69f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:16.625955 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:16.633666 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:16.673651 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0b82e8-6198-4575-9cc6-847324a16ef4 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:16.681264 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d0c869-54f3-4a8e-837c-74979cc1f69f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:16.706329 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f620fbfd-699e-4228-86d9-feee9f9e13ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:16.707365 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f620fbfd-699e-4228-86d9-feee9f9e13ce None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:16.707829 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f620fbfd-699e-4228-86d9-feee9f9e13ce None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:16.708111 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f620fbfd-699e-4228-86d9-feee9f9e13ce None admin] PATH_INFO: `/v3/users/nova` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:16.715577 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef47678-6c8c-4ffd-9684-5d06c26cf130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:16.715718 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f620fbfd-699e-4228-86d9-feee9f9e13ce None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:16.716356 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef47678-6c8c-4ffd-9684-5d06c26cf130 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:16.716640 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef47678-6c8c-4ffd-9684-5d06c26cf130 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:16.716918 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef47678-6c8c-4ffd-9684-5d06c26cf130 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:16.717131 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f620fbfd-699e-4228-86d9-feee9f9e13ce None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:16.727154 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef47678-6c8c-4ffd-9684-5d06c26cf130 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:16.728753 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef47678-6c8c-4ffd-9684-5d06c26cf130 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:16.731683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-f620fbfd-699e-4228-86d9-feee9f9e13ce None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 15:01:16.731683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:16.731683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:16.731683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:16.731683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:16.731683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:16.731683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:16.731683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:16.731683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:16.731683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:16.731683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:16.731683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:16.731683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 15:01:16.731683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 15:01:16.731683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 15:01:16.731683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 15:01:16.732616 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:16.732616 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:16.732616 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 15:01:16.732616 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:16.732616 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 15:01:16.732616 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:16.732616 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:16.732616 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:16.732616 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:16.732616 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:16.732616 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:16.732616 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:16.732616 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:16.732616 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:16.732616 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:16.732616 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:16.732616 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:16.732616 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:16.733566 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:16.733566 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:16.733566 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 15:01:16.733566 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 15:01:16.733566 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 15:01:16.733566 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 15:01:16.733566 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 15:01:16.733566 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 15:01:16.733566 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 15:01:16.733566 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:16.733566 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 285/574] 10.222.0.50 () {56 vars in 1093 bytes} [Fri May 27 15:01:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:16.734194 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f620fbfd-699e-4228-86d9-feee9f9e13ce None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:16.746897 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-cef47678-6c8c-4ffd-9684-5d06c26cf130 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 27 15:01:16.746897 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:16.746897 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:16.746897 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:16.746897 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:16.746897 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:16.746897 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:16.746897 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:16.746897 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:16.746897 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:16.746897 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:16.746897 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:16.746897 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 15:01:16.746897 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 15:01:16.746897 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 15:01:16.746897 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 15:01:16.758801 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:16.758801 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:16.758801 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 15:01:16.758801 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:16.758801 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 15:01:16.758801 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:16.758801 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:16.758801 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:16.758801 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:16.758801 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:16.758801 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:16.758801 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:16.758801 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:16.758801 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:16.758801 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:16.758801 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:16.758801 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:16.758801 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:16.760064 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:16.760064 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:16.760064 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 15:01:16.760064 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 15:01:16.760064 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 15:01:16.760064 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 15:01:16.760064 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 15:01:16.760064 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 15:01:16.760064 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 27 15:01:16.760064 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:16.760064 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 290/575] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:01:16 2022] GET /identity/v3/users/glance => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:16.760064 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef47678-6c8c-4ffd-9684-5d06c26cf130 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:16.841062 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f620fbfd-699e-4228-86d9-feee9f9e13ce None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:16.857080 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef47678-6c8c-4ffd-9684-5d06c26cf130 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:16.858596 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:16.864066 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:16.875767 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f40e5-f95d-4e77-a1e3-6e53fca5df6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:16.876564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f40e5-f95d-4e77-a1e3-6e53fca5df6a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:16.876845 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f40e5-f95d-4e77-a1e3-6e53fca5df6a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:16.877121 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f40e5-f95d-4e77-a1e3-6e53fca5df6a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:16.887844 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802f40e5-f95d-4e77-a1e3-6e53fca5df6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:01:16.892703 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b9c04f-583e-4080-9063-63dfbd4cad41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:16.893354 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b9c04f-583e-4080-9063-63dfbd4cad41 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:16.893642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b9c04f-583e-4080-9063-63dfbd4cad41 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:16.893923 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b9c04f-583e-4080-9063-63dfbd4cad41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:16.894756 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b9c04f-583e-4080-9063-63dfbd4cad41 None admin] RBAC: Adding query filter params (role.id=13f37584013544e7ad1e2c8f350da6d2, scope.project.id=daf12585428b41a2b45bdbc85413a6c4, user.id=18903f2eab7c415096e075d47f0b9a4d) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:16.895102 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b9c04f-583e-4080-9063-63dfbd4cad41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:16.896524 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b9c04f-583e-4080-9063-63dfbd4cad41 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:16.896743 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-802f40e5-f95d-4e77-a1e3-6e53fca5df6a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 15:01:16.896743 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:16.896743 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:16.896743 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:16.896743 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:16.896743 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:16.896743 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:16.896743 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:16.896743 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:16.896743 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:16.896743 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:16.896743 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:16.896743 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:01:16.896743 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:01:16.896743 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:01:16.896743 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:01:16.897801 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:01:16.897801 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:01:16.897801 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:01:16.897801 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:01:16.897801 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:16.897801 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:16.897801 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:16.897801 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:16.897801 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:16.897801 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:16.897801 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:16.897801 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:16.897801 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:16.897801 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:16.897801 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:16.897801 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:16.898872 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:16.898872 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:16.898872 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:16.898872 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:16.898872 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:01:16.898872 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:01:16.898872 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:01:16.898872 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:01:16.898872 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:01:16.898872 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:01:16.898872 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 15:01:16.898872 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:16.898872 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 286/576] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 15:01:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:16.899766 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f40e5-f95d-4e77-a1e3-6e53fca5df6a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:16.907321 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 291/577] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:01:16 2022] GET /identity/v3/role_assignments?user.id=18903f2eab7c415096e075d47f0b9a4d&scope.project.id=daf12585428b41a2b45bdbc85413a6c4&role.id=13f37584013544e7ad1e2c8f350da6d2 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:16.908128 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b9c04f-583e-4080-9063-63dfbd4cad41 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:17.021612 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f40e5-f95d-4e77-a1e3-6e53fca5df6a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:17.044237 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b9c04f-583e-4080-9063-63dfbd4cad41 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:17.059960 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f0a2b-f0aa-4f3a-8a6a-7872928ecd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:17.060700 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f0a2b-f0aa-4f3a-8a6a-7872928ecd75 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:17.060906 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f0a2b-f0aa-4f3a-8a6a-7872928ecd75 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:17.061186 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616f0a2b-f0aa-4f3a-8a6a-7872928ecd75 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:17.062560 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616f0a2b-f0aa-4f3a-8a6a-7872928ecd75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:17.062817 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616f0a2b-f0aa-4f3a-8a6a-7872928ecd75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:17.064187 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616f0a2b-f0aa-4f3a-8a6a-7872928ecd75 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:17.078483 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 287/578] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:17.079383 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f0a2b-f0aa-4f3a-8a6a-7872928ecd75 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:17.090836 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554009c3-e29f-4863-99e1-61fcc4c4f103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:17.091900 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554009c3-e29f-4863-99e1-61fcc4c4f103 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:17.092121 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554009c3-e29f-4863-99e1-61fcc4c4f103 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:17.092393 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554009c3-e29f-4863-99e1-61fcc4c4f103 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:17.093725 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554009c3-e29f-4863-99e1-61fcc4c4f103 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:17.093845 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554009c3-e29f-4863-99e1-61fcc4c4f103 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:17.095210 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554009c3-e29f-4863-99e1-61fcc4c4f103 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:17.115264 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 292/579] 10.222.0.50 () {56 vars in 1133 bytes} [Fri May 27 15:01:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:17.115264 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554009c3-e29f-4863-99e1-61fcc4c4f103 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:17.134313 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:17.142765 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:17.200128 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616f0a2b-f0aa-4f3a-8a6a-7872928ecd75 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:17.232552 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554009c3-e29f-4863-99e1-61fcc4c4f103 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:17.234828 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507895b2-4e1d-4a90-bd4a-a339b08a597f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:17.235526 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507895b2-4e1d-4a90-bd4a-a339b08a597f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:17.235804 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507895b2-4e1d-4a90-bd4a-a339b08a597f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:17.236077 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507895b2-4e1d-4a90-bd4a-a339b08a597f None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:17.236907 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507895b2-4e1d-4a90-bd4a-a339b08a597f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:17.237213 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507895b2-4e1d-4a90-bd4a-a339b08a597f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:17.238647 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507895b2-4e1d-4a90-bd4a-a339b08a597f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:17.260121 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 288/580] 10.222.0.50 () {56 vars in 1137 bytes} [Fri May 27 15:01:17 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:17.260957 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507895b2-4e1d-4a90-bd4a-a339b08a597f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:17.274154 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b5f1ea-5b60-474c-b3bf-807f7a88bc60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:17.274829 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5f1ea-5b60-474c-b3bf-807f7a88bc60 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:17.275144 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5f1ea-5b60-474c-b3bf-807f7a88bc60 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:17.275423 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5f1ea-5b60-474c-b3bf-807f7a88bc60 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:17.276170 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b5f1ea-5b60-474c-b3bf-807f7a88bc60 None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:17.276482 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b5f1ea-5b60-474c-b3bf-807f7a88bc60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:17.277847 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b5f1ea-5b60-474c-b3bf-807f7a88bc60 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:17.286445 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 293/581] 10.222.0.50 () {56 vars in 1103 bytes} [Fri May 27 15:01:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:17.287305 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b5f1ea-5b60-474c-b3bf-807f7a88bc60 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:17.366053 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:17.369356 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507895b2-4e1d-4a90-bd4a-a339b08a597f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:17.380667 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:17.401714 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b5f1ea-5b60-474c-b3bf-807f7a88bc60 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:17.409931 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b847c7ca-789b-43ec-839f-d211817cb298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:17.411224 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b847c7ca-789b-43ec-839f-d211817cb298 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:17.411520 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b847c7ca-789b-43ec-839f-d211817cb298 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:17.411800 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b847c7ca-789b-43ec-839f-d211817cb298 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:17.420931 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b847c7ca-789b-43ec-839f-d211817cb298 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:17.422936 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b847c7ca-789b-43ec-839f-d211817cb298 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:17.436432 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c18b92-c00e-4288-9599-2e5dbdf9792b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:17.437916 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c18b92-c00e-4288-9599-2e5dbdf9792b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:17.438171 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-b847c7ca-789b-43ec-839f-d211817cb298 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 27 15:01:17.438171 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:17.438171 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:17.438171 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:17.438171 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:17.438171 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:17.438171 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:17.438171 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:17.438171 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:17.438171 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:17.438171 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:17.438171 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:17.438171 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 15:01:17.438171 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 15:01:17.438171 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 15:01:17.438171 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 15:01:17.439413 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:17.439413 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:17.439413 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 15:01:17.439413 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:17.439413 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 15:01:17.439413 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:17.439413 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:17.439413 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:17.439413 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:17.439413 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:17.439413 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:17.439413 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:17.439413 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:17.439413 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:17.439413 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:17.439413 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:17.439413 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:17.439413 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:17.440360 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:17.440360 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:17.440360 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 15:01:17.440360 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 15:01:17.440360 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 15:01:17.440360 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 15:01:17.440360 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 15:01:17.440360 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 15:01:17.440360 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 27 15:01:17.440360 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:17.440897 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c18b92-c00e-4288-9599-2e5dbdf9792b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:17.440897 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c18b92-c00e-4288-9599-2e5dbdf9792b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:17.441035 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 289/582] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:01:17 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:17.441035 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b847c7ca-789b-43ec-839f-d211817cb298 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:17.452554 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c18b92-c00e-4288-9599-2e5dbdf9792b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:17.453697 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c18b92-c00e-4288-9599-2e5dbdf9792b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:17.471153 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-52c18b92-c00e-4288-9599-2e5dbdf9792b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:17.471153 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:17.471153 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:17.471153 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:17.471153 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:17.471153 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:17.471153 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:17.471153 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:17.471153 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:17.471153 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:17.471153 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:17.471153 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:17.471153 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:17.471153 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:17.471153 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:17.472080 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:17.472080 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:17.472080 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:17.472080 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:17.472080 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:17.472080 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:17.472080 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:17.472080 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:17.472080 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:17.472080 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:17.472080 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:17.472080 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:17.472080 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:17.472080 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:17.472080 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:17.472080 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:17.472080 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:17.472998 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:17.472998 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:17.472998 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:17.472998 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:17.472998 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 294/583] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:17.473304 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c18b92-c00e-4288-9599-2e5dbdf9792b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:17.553889 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b847c7ca-789b-43ec-839f-d211817cb298 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:17.579820 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c18b92-c00e-4288-9599-2e5dbdf9792b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:17.589938 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48ff64-1dee-483d-aa81-0b4f07346a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:17.590879 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48ff64-1dee-483d-aa81-0b4f07346a62 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:17.591165 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48ff64-1dee-483d-aa81-0b4f07346a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:17.591442 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48ff64-1dee-483d-aa81-0b4f07346a62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:17.602780 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f48ff64-1dee-483d-aa81-0b4f07346a62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:17.604263 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f48ff64-1dee-483d-aa81-0b4f07346a62 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:17.613633 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97664a35-0d0a-4590-b775-f0f834e7aed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:17.614431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97664a35-0d0a-4590-b775-f0f834e7aed3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:17.614714 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97664a35-0d0a-4590-b775-f0f834e7aed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:17.615024 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97664a35-0d0a-4590-b775-f0f834e7aed3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:17.621740 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:17.626095 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-3f48ff64-1dee-483d-aa81-0b4f07346a62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:17.626095 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:17.626095 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:17.626095 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:17.626095 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:17.626095 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:17.626095 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:17.626095 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:17.626095 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:17.626095 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:17.626095 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:17.626095 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:17.626095 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:17.626095 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:17.626095 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:17.627176 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:17.627176 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:17.627176 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:17.627176 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:17.627176 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:17.627176 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:17.627176 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:17.627176 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:17.627176 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:17.627176 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:17.627176 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:17.627176 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:17.627176 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:17.627176 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:17.627176 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:17.627176 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:17.627176 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:17.628093 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:17.628093 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:17.628093 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:17.628093 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:17.628093 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 290/584] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:17.628093 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48ff64-1dee-483d-aa81-0b4f07346a62 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:17.632447 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97664a35-0d0a-4590-b775-f0f834e7aed3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:17.633921 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97664a35-0d0a-4590-b775-f0f834e7aed3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:17.644010 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:17.652010 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-97664a35-0d0a-4590-b775-f0f834e7aed3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:17.652010 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:17.652010 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:17.652010 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:17.652010 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:17.652010 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:17.652010 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:17.652010 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:17.652010 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:17.652010 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:17.652010 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:17.652010 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:17.652010 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:17.652010 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:17.652010 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:17.652896 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:17.652896 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:17.652896 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:17.652896 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:17.652896 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:17.652896 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:17.652896 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:17.652896 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:17.652896 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:17.652896 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:17.652896 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:17.652896 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:17.652896 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:17.652896 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:17.652896 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:17.652896 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:17.652896 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:17.653805 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:17.653805 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:17.653805 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:17.653805 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:17.657981 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 295/585] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:17.658636 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97664a35-0d0a-4590-b775-f0f834e7aed3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:17.735843 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48ff64-1dee-483d-aa81-0b4f07346a62 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:17.760522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97664a35-0d0a-4590-b775-f0f834e7aed3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:17.770913 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2458fefa-df77-4d16-aeaa-418313f53825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:17.771700 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458fefa-df77-4d16-aeaa-418313f53825 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:17.771861 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458fefa-df77-4d16-aeaa-418313f53825 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:17.772124 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458fefa-df77-4d16-aeaa-418313f53825 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:17.773000 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2458fefa-df77-4d16-aeaa-418313f53825 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:17.773310 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2458fefa-df77-4d16-aeaa-418313f53825 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:17.774747 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2458fefa-df77-4d16-aeaa-418313f53825 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:17.792665 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 291/586] 10.222.0.50 () {56 vars in 1137 bytes} [Fri May 27 15:01:17 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:17.793290 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2458fefa-df77-4d16-aeaa-418313f53825 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:17.800649 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23da60f-4278-4b17-9e32-9d3680c2577a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:17.801617 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23da60f-4278-4b17-9e32-9d3680c2577a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:17.802569 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23da60f-4278-4b17-9e32-9d3680c2577a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:17.803102 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23da60f-4278-4b17-9e32-9d3680c2577a None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:17.804501 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23da60f-4278-4b17-9e32-9d3680c2577a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:17.805021 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23da60f-4278-4b17-9e32-9d3680c2577a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:17.807265 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23da60f-4278-4b17-9e32-9d3680c2577a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:17.820660 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 296/587] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:17.821656 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23da60f-4278-4b17-9e32-9d3680c2577a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:17.881078 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:17.888678 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:17.922347 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2458fefa-df77-4d16-aeaa-418313f53825 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:17.941796 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23da60f-4278-4b17-9e32-9d3680c2577a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:17.965933 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cb5ce2-119f-4397-9b92-acbf761db03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:17.966812 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb5ce2-119f-4397-9b92-acbf761db03e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:17.966963 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb5ce2-119f-4397-9b92-acbf761db03e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:17.967228 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb5ce2-119f-4397-9b92-acbf761db03e None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:17.968444 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cb5ce2-119f-4397-9b92-acbf761db03e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:17.968740 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cb5ce2-119f-4397-9b92-acbf761db03e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:17.970110 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cb5ce2-119f-4397-9b92-acbf761db03e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:17.978163 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c0d56a-982d-4f81-a1d6-a183c545b356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:17.978831 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0d56a-982d-4f81-a1d6-a183c545b356 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:17.979139 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0d56a-982d-4f81-a1d6-a183c545b356 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:17.979417 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0d56a-982d-4f81-a1d6-a183c545b356 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:17.980134 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c0d56a-982d-4f81-a1d6-a183c545b356 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:17.980440 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c0d56a-982d-4f81-a1d6-a183c545b356 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:17.986807 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c0d56a-982d-4f81-a1d6-a183c545b356 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:17.986964 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 292/588] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:17.986964 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cb5ce2-119f-4397-9b92-acbf761db03e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:17.999375 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 297/589] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:18.000390 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c0d56a-982d-4f81-a1d6-a183c545b356 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:18.111920 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c0d56a-982d-4f81-a1d6-a183c545b356 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:18.114406 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cb5ce2-119f-4397-9b92-acbf761db03e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:18.122572 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:18.142827 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:18.148396 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec9336-9e99-48c3-8d0e-cfad2a21fb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:18.149202 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec9336-9e99-48c3-8d0e-cfad2a21fb22 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:18.149548 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec9336-9e99-48c3-8d0e-cfad2a21fb22 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:18.149829 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec9336-9e99-48c3-8d0e-cfad2a21fb22 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:18.153029 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48165faa-e0d2-427a-921b-db2c539daa62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:18.157226 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48165faa-e0d2-427a-921b-db2c539daa62 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:18.157226 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48165faa-e0d2-427a-921b-db2c539daa62 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:18.157226 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48165faa-e0d2-427a-921b-db2c539daa62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:18.158780 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ec9336-9e99-48c3-8d0e-cfad2a21fb22 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:18.160263 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ec9336-9e99-48c3-8d0e-cfad2a21fb22 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:18.170371 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48165faa-e0d2-427a-921b-db2c539daa62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:18.172436 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48165faa-e0d2-427a-921b-db2c539daa62 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:18.175522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-85ec9336-9e99-48c3-8d0e-cfad2a21fb22 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:18.175522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:18.175522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:18.175522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:18.175522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:18.175522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:18.175522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:18.175522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:18.175522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:18.175522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:18.175522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:18.175522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:18.175522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 15:01:18.175522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 15:01:18.175522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 15:01:18.176473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 15:01:18.176473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:18.176473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:18.176473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:18.176473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:18.176473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:18.176473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:18.176473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:18.176473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:18.176473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:18.176473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:18.176473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:18.176473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:18.176473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:18.176473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:18.176473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:18.176473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:18.182291 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 15:01:18.182291 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 15:01:18.182291 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 15:01:18.182291 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 15:01:18.182291 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 15:01:18.182291 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 15:01:18.182291 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:18.182291 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:18.182291 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 298/590] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:01:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:18.182291 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec9336-9e99-48c3-8d0e-cfad2a21fb22 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:18.194665 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-48165faa-e0d2-427a-921b-db2c539daa62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:18.194665 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:18.194665 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:18.194665 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:18.194665 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:18.194665 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:18.194665 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:18.194665 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:18.194665 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:18.194665 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:18.194665 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:18.194665 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:18.194665 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:18.194665 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:18.194665 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:18.195639 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:18.195639 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:18.195639 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:18.195639 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:18.195639 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:18.195639 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:18.195639 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:18.195639 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:18.195639 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:18.195639 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:18.195639 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:18.195639 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:18.195639 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:18.195639 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:18.195639 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:18.195639 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:18.195639 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:18.196664 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:18.196664 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:18.196664 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:18.196664 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:18.196664 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 293/591] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:18.197302 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48165faa-e0d2-427a-921b-db2c539daa62 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:18.292195 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec9336-9e99-48c3-8d0e-cfad2a21fb22 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:18.299725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48165faa-e0d2-427a-921b-db2c539daa62 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:18.330735 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb343931-050d-410b-8654-e87ac826352f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:18.331558 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb343931-050d-410b-8654-e87ac826352f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:18.331845 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb343931-050d-410b-8654-e87ac826352f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:18.332121 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb343931-050d-410b-8654-e87ac826352f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:18.333883 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35079f03-8036-4429-a30e-4e3554726603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:18.334667 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35079f03-8036-4429-a30e-4e3554726603 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:18.334980 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35079f03-8036-4429-a30e-4e3554726603 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:18.335256 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35079f03-8036-4429-a30e-4e3554726603 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:18.340407 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb343931-050d-410b-8654-e87ac826352f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:18.342206 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb343931-050d-410b-8654-e87ac826352f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:18.343849 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35079f03-8036-4429-a30e-4e3554726603 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:18.345437 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35079f03-8036-4429-a30e-4e3554726603 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:18.358667 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-fb343931-050d-410b-8654-e87ac826352f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:18.358667 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:18.358667 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:18.358667 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:18.358667 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:18.358667 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:18.358667 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:18.358667 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:18.358667 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:18.358667 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:18.358667 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:18.358667 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:18.358667 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 15:01:18.358667 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 15:01:18.358667 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 15:01:18.359650 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 15:01:18.359650 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:18.359650 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:18.359650 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:18.359650 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:18.359650 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:18.359650 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:18.359650 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:18.359650 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:18.359650 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:18.359650 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:18.359650 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:18.359650 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:18.359650 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:18.359650 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:18.359650 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:18.359650 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:18.360575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 15:01:18.360575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 15:01:18.360575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 15:01:18.360575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 15:01:18.360575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 15:01:18.360575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 15:01:18.360575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:18.360575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:18.361013 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-35079f03-8036-4429-a30e-4e3554726603 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 27 15:01:18.361013 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:18.361013 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:18.361013 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:18.361013 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:18.361013 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:18.361013 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:18.361013 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:18.361013 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:18.361013 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:18.361013 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:18.361013 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:18.361013 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 15:01:18.361013 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 15:01:18.361013 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 15:01:18.361013 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 15:01:18.366982 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:18.366982 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:18.366982 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 15:01:18.366982 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:18.366982 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 15:01:18.366982 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:18.366982 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:18.366982 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:18.366982 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:18.366982 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:18.366982 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:18.366982 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:18.366982 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:18.366982 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:18.366982 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:18.366982 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:18.366982 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:18.366982 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:18.368837 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:18.368837 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:18.368837 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 15:01:18.368837 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 15:01:18.368837 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 15:01:18.368837 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 15:01:18.368837 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 15:01:18.368837 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 15:01:18.368837 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 27 15:01:18.368837 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:18.368837 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 294/592] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 15:01:18 2022] GET /identity/v3/users/neutron => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:18.368837 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35079f03-8036-4429-a30e-4e3554726603 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:18.370936 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 299/593] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:01:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:18.370936 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb343931-050d-410b-8654-e87ac826352f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:18.396018 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:18.415977 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:18.473140 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb343931-050d-410b-8654-e87ac826352f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:18.477619 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35079f03-8036-4429-a30e-4e3554726603 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:18.509461 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d976065-af4b-49d1-9025-3ad361378afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:18.510442 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d976065-af4b-49d1-9025-3ad361378afd None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:18.510984 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d976065-af4b-49d1-9025-3ad361378afd None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:18.511109 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7252fd59-f883-4f84-b51c-e4b7c33be619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:18.511619 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d976065-af4b-49d1-9025-3ad361378afd None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:18.511732 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252fd59-f883-4f84-b51c-e4b7c33be619 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:18.512016 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252fd59-f883-4f84-b51c-e4b7c33be619 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:18.512287 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7252fd59-f883-4f84-b51c-e4b7c33be619 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:18.513057 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7252fd59-f883-4f84-b51c-e4b7c33be619 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:18.513303 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d976065-af4b-49d1-9025-3ad361378afd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:18.513412 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7252fd59-f883-4f84-b51c-e4b7c33be619 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:18.513910 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d976065-af4b-49d1-9025-3ad361378afd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:18.514767 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7252fd59-f883-4f84-b51c-e4b7c33be619 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:18.516208 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d976065-af4b-49d1-9025-3ad361378afd None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:18.527528 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 295/594] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 15:01:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:18.528322 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7252fd59-f883-4f84-b51c-e4b7c33be619 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:18.531952 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 300/595] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:18.533025 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d976065-af4b-49d1-9025-3ad361378afd None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:18.633309 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:18.634123 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7252fd59-f883-4f84-b51c-e4b7c33be619 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:18.636024 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:18.643681 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d976065-af4b-49d1-9025-3ad361378afd None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:18.672825 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c411cbc-ffa1-4200-b29e-d73f1721fabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:18.673429 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c411cbc-ffa1-4200-b29e-d73f1721fabd None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:18.673682 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c411cbc-ffa1-4200-b29e-d73f1721fabd None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:18.673956 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c411cbc-ffa1-4200-b29e-d73f1721fabd None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:18.674728 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c411cbc-ffa1-4200-b29e-d73f1721fabd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:18.675067 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c411cbc-ffa1-4200-b29e-d73f1721fabd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:18.676486 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c411cbc-ffa1-4200-b29e-d73f1721fabd None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:18.680015 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197502a7-232f-4849-8f9c-1a4c11afbcc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:18.680684 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197502a7-232f-4849-8f9c-1a4c11afbcc2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:18.680990 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197502a7-232f-4849-8f9c-1a4c11afbcc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:18.681267 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197502a7-232f-4849-8f9c-1a4c11afbcc2 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:18.682096 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197502a7-232f-4849-8f9c-1a4c11afbcc2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:18.682404 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197502a7-232f-4849-8f9c-1a4c11afbcc2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:18.683833 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197502a7-232f-4849-8f9c-1a4c11afbcc2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:18.689446 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 296/596] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 15:01:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:18.694631 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c411cbc-ffa1-4200-b29e-d73f1721fabd None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:18.707102 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 301/597] 10.222.0.50 () {56 vars in 1139 bytes} [Fri May 27 15:01:18 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:18.707102 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197502a7-232f-4849-8f9c-1a4c11afbcc2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:18.806466 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197502a7-232f-4849-8f9c-1a4c11afbcc2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:18.808566 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c411cbc-ffa1-4200-b29e-d73f1721fabd None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:18.844963 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352cd1f8-8bd0-4d74-b0fb-cf2b4fa4a0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:18.845758 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352cd1f8-8bd0-4d74-b0fb-cf2b4fa4a0a0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:18.846038 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352cd1f8-8bd0-4d74-b0fb-cf2b4fa4a0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:18.846334 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352cd1f8-8bd0-4d74-b0fb-cf2b4fa4a0a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:18.848574 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b826fb0-aa72-47c2-80e0-26c942725c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:18.849453 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b826fb0-aa72-47c2-80e0-26c942725c7d None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:18.849731 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b826fb0-aa72-47c2-80e0-26c942725c7d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:18.850001 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b826fb0-aa72-47c2-80e0-26c942725c7d None admin] PATH_INFO: `/v3/projects/daf12585428b41a2b45bdbc85413a6c4/users/451a9232f95e494683d81b6078a318d0/roles/13f37584013544e7ad1e2c8f350da6d2` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:18.853267 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352cd1f8-8bd0-4d74-b0fb-cf2b4fa4a0a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:18.854706 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352cd1f8-8bd0-4d74-b0fb-cf2b4fa4a0a0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:18.865392 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b826fb0-aa72-47c2-80e0-26c942725c7d None admin] RBAC: Authorizing `identity:create_grant(project_id=daf12585428b41a2b45bdbc85413a6c4, user_id=451a9232f95e494683d81b6078a318d0, role_id=13f37584013544e7ad1e2c8f350da6d2)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:18.866836 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b826fb0-aa72-47c2-80e0-26c942725c7d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:18.869312 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-352cd1f8-8bd0-4d74-b0fb-cf2b4fa4a0a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:18.869312 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:18.869312 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:18.869312 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:18.869312 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:18.869312 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:18.869312 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:18.869312 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:18.869312 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:18.869312 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:18.869312 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:18.869312 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:18.869312 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 15:01:18.869312 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 15:01:18.869312 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 15:01:18.870751 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 15:01:18.870751 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:18.870751 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:18.870751 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:18.870751 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:18.870751 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:18.870751 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:18.870751 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:18.870751 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:18.870751 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:18.870751 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:18.870751 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:18.870751 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:18.870751 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:18.870751 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:18.870751 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:18.870751 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:18.872017 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 15:01:18.872017 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 15:01:18.872017 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 15:01:18.872017 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 15:01:18.872017 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 15:01:18.872017 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 15:01:18.872017 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:18.872017 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:18.872017 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 302/598] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:01:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:18.872017 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352cd1f8-8bd0-4d74-b0fb-cf2b4fa4a0a0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:18.885287 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:18.888806 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 297/599] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 15:01:18 2022] PUT /identity/v3/projects/daf12585428b41a2b45bdbc85413a6c4/users/451a9232f95e494683d81b6078a318d0/roles/13f37584013544e7ad1e2c8f350da6d2 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:01:18.899096 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b826fb0-aa72-47c2-80e0-26c942725c7d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:18.917878 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:19.014994 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352cd1f8-8bd0-4d74-b0fb-cf2b4fa4a0a0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:19.056901 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94adf3fd-d4d7-49ca-8dff-0d10341c7dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:19.057412 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b826fb0-aa72-47c2-80e0-26c942725c7d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:19.057836 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94adf3fd-d4d7-49ca-8dff-0d10341c7dc0 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:19.058116 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94adf3fd-d4d7-49ca-8dff-0d10341c7dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:19.058391 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94adf3fd-d4d7-49ca-8dff-0d10341c7dc0 None admin] PATH_INFO: `/v3/projects/daf12585428b41a2b45bdbc85413a6c4/users/1afda9d6b1d64e6ab400b26124cf1290/roles/13f37584013544e7ad1e2c8f350da6d2` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:19.076302 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94adf3fd-d4d7-49ca-8dff-0d10341c7dc0 None admin] RBAC: Authorizing `identity:create_grant(project_id=daf12585428b41a2b45bdbc85413a6c4, user_id=1afda9d6b1d64e6ab400b26124cf1290, role_id=13f37584013544e7ad1e2c8f350da6d2)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:19.078084 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94adf3fd-d4d7-49ca-8dff-0d10341c7dc0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:19.102873 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 303/600] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 15:01:18 2022] PUT /identity/v3/projects/daf12585428b41a2b45bdbc85413a6c4/users/1afda9d6b1d64e6ab400b26124cf1290/roles/13f37584013544e7ad1e2c8f350da6d2 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:01:19.102873 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94adf3fd-d4d7-49ca-8dff-0d10341c7dc0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:19.109410 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa35edae-4b3e-4fdb-8301-58c857172fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:19.110274 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa35edae-4b3e-4fdb-8301-58c857172fb5 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:19.110556 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa35edae-4b3e-4fdb-8301-58c857172fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:19.110827 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa35edae-4b3e-4fdb-8301-58c857172fb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:19.123249 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa35edae-4b3e-4fdb-8301-58c857172fb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:19.125609 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa35edae-4b3e-4fdb-8301-58c857172fb5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:19.150009 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-fa35edae-4b3e-4fdb-8301-58c857172fb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:19.150009 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:19.150009 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:19.150009 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:19.150009 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:19.150009 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:19.150009 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:19.150009 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:19.150009 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:19.150009 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:19.150009 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:19.150009 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:19.150009 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:19.150009 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:19.150009 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:19.151186 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:19.151186 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:19.151186 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:19.151186 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:19.151186 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:19.151186 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:19.151186 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:19.151186 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:19.151186 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:19.151186 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:19.151186 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:19.151186 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:19.151186 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:19.151186 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:19.151186 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:19.151186 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:19.151186 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:19.152107 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:19.152107 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:19.152107 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:19.152107 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:19.152107 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 298/601] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:19.164983 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa35edae-4b3e-4fdb-8301-58c857172fb5 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:19.179124 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:19.227321 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94adf3fd-d4d7-49ca-8dff-0d10341c7dc0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:19.256743 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:19.267912 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88617d98-839c-4142-b7c8-7c50d020133e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:19.268494 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88617d98-839c-4142-b7c8-7c50d020133e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:19.268773 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88617d98-839c-4142-b7c8-7c50d020133e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:19.269045 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88617d98-839c-4142-b7c8-7c50d020133e None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:19.269834 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88617d98-839c-4142-b7c8-7c50d020133e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:19.270137 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88617d98-839c-4142-b7c8-7c50d020133e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:19.271604 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88617d98-839c-4142-b7c8-7c50d020133e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:19.286492 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 304/602] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 15:01:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:19.287768 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee45a4fd-612c-4693-8786-feb9348e4ca7 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:19.288028 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee45a4fd-612c-4693-8786-feb9348e4ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:19.288238 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee45a4fd-612c-4693-8786-feb9348e4ca7 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:19.289795 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 305/603] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:19.295805 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d83b3d5-b6a9-42e4-9009-4504cebaeccf None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:19.295805 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d83b3d5-b6a9-42e4-9009-4504cebaeccf None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:19.295805 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d83b3d5-b6a9-42e4-9009-4504cebaeccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:19.316915 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1d83b3d5-b6a9-42e4-9009-4504cebaeccf None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:19.317647 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa35edae-4b3e-4fdb-8301-58c857172fb5 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:19.318585 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1d83b3d5-b6a9-42e4-9009-4504cebaeccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:19.363728 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc680e-d4c5-4c55-8bea-b9c6762b02f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:19.364561 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc680e-d4c5-4c55-8bea-b9c6762b02f4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:19.364937 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc680e-d4c5-4c55-8bea-b9c6762b02f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:19.365313 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc680e-d4c5-4c55-8bea-b9c6762b02f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:19.366133 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc680e-d4c5-4c55-8bea-b9c6762b02f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:19.366533 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc680e-d4c5-4c55-8bea-b9c6762b02f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:19.368186 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc680e-d4c5-4c55-8bea-b9c6762b02f4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:19.382956 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 299/604] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:19.382956 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc680e-d4c5-4c55-8bea-b9c6762b02f4 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:19.407527 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 306/605] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 15:01:19.408401 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d83b3d5-b6a9-42e4-9009-4504cebaeccf None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:19.481457 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:19.516592 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc680e-d4c5-4c55-8bea-b9c6762b02f4 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:19.542549 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:19.555279 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3e99f-ef3e-4916-aebe-a69232a53afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:19.555488 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d83b3d5-b6a9-42e4-9009-4504cebaeccf None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:19.556180 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3e99f-ef3e-4916-aebe-a69232a53afa None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:19.556466 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3e99f-ef3e-4916-aebe-a69232a53afa None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:19.556750 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3e99f-ef3e-4916-aebe-a69232a53afa None admin] PATH_INFO: `/v3/projects/daf12585428b41a2b45bdbc85413a6c4/users/896af6ccf89847e5903f6a4fe3d9d810/roles/13f37584013544e7ad1e2c8f350da6d2` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:19.572921 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e3e99f-ef3e-4916-aebe-a69232a53afa None admin] RBAC: Authorizing `identity:create_grant(project_id=daf12585428b41a2b45bdbc85413a6c4, user_id=896af6ccf89847e5903f6a4fe3d9d810, role_id=13f37584013544e7ad1e2c8f350da6d2)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:19.574422 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e3e99f-ef3e-4916-aebe-a69232a53afa None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:19.596254 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 300/606] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 15:01:19 2022] PUT /identity/v3/projects/daf12585428b41a2b45bdbc85413a6c4/users/896af6ccf89847e5903f6a4fe3d9d810/roles/13f37584013544e7ad1e2c8f350da6d2 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:01:19.597408 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3e99f-ef3e-4916-aebe-a69232a53afa None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:19.598208 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6812aca8-8d55-4564-842d-fe968a292d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:19.599267 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6812aca8-8d55-4564-842d-fe968a292d0e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:19.599678 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6812aca8-8d55-4564-842d-fe968a292d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:19.600141 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6812aca8-8d55-4564-842d-fe968a292d0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:19.609801 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6812aca8-8d55-4564-842d-fe968a292d0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:19.611662 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6812aca8-8d55-4564-842d-fe968a292d0e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:19.628715 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-6812aca8-8d55-4564-842d-fe968a292d0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:19.628715 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:19.628715 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:19.628715 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:19.628715 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:19.628715 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:19.628715 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:19.628715 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:19.628715 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:19.628715 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:19.628715 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:19.628715 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:19.628715 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 15:01:19.628715 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 15:01:19.628715 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 15:01:19.629695 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 15:01:19.629695 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:19.629695 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:19.629695 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:19.629695 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:19.629695 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:19.629695 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:19.629695 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:19.629695 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:19.629695 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:19.629695 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:19.629695 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:19.629695 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:19.629695 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:19.629695 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:19.629695 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:19.629695 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:19.631267 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 15:01:19.631267 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 15:01:19.631267 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 15:01:19.631267 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 15:01:19.631267 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 15:01:19.631267 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 15:01:19.631267 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:19.631267 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:19.631267 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 307/607] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:01:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:19.635956 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6812aca8-8d55-4564-842d-fe968a292d0e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:19.721701 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3e99f-ef3e-4916-aebe-a69232a53afa None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:19.767010 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37888b43-1ac9-4ef7-800c-3848c7eb0d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:19.767835 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37888b43-1ac9-4ef7-800c-3848c7eb0d02 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:19.768125 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37888b43-1ac9-4ef7-800c-3848c7eb0d02 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:19.768405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37888b43-1ac9-4ef7-800c-3848c7eb0d02 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:19.782125 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37888b43-1ac9-4ef7-800c-3848c7eb0d02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:01:19.791024 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-37888b43-1ac9-4ef7-800c-3848c7eb0d02 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 15:01:19.791024 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:19.791024 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:19.791024 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:19.791024 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:19.791024 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:19.791024 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:19.791024 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:19.791024 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:19.791024 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:19.791024 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:19.791024 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:19.791024 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:01:19.791024 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:01:19.791024 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:01:19.791024 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:01:19.793405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:01:19.793405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:01:19.793405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:01:19.793405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:01:19.793405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:19.793405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:19.793405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:19.793405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:19.793405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:19.793405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:19.793405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:19.793405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:19.793405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:19.793405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:19.793405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:19.793405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:19.795676 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:19.795676 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:19.795676 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:19.795676 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:19.795676 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:01:19.795676 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:01:19.795676 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:01:19.795676 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:01:19.795676 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:01:19.795676 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:01:19.795676 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 15:01:19.795676 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:19.795676 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 301/608] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 15:01:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:19.804254 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37888b43-1ac9-4ef7-800c-3848c7eb0d02 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:19.854218 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:19.861356 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6812aca8-8d55-4564-842d-fe968a292d0e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:19.885239 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:19.923157 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808b6c8f-1213-4d3b-84d4-a777afce0a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:19.923828 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b6c8f-1213-4d3b-84d4-a777afce0a2f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:19.924113 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b6c8f-1213-4d3b-84d4-a777afce0a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:19.924394 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b6c8f-1213-4d3b-84d4-a777afce0a2f None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:19.925363 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808b6c8f-1213-4d3b-84d4-a777afce0a2f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:19.926073 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808b6c8f-1213-4d3b-84d4-a777afce0a2f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:19.927575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808b6c8f-1213-4d3b-84d4-a777afce0a2f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:19.943769 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 308/609] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 15:01:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:19.949344 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808b6c8f-1213-4d3b-84d4-a777afce0a2f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:19.960354 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37888b43-1ac9-4ef7-800c-3848c7eb0d02 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:20.008230 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3346c0ce-a028-4630-bc8d-f6ad2815f98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:20.009065 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3346c0ce-a028-4630-bc8d-f6ad2815f98b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:20.009400 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3346c0ce-a028-4630-bc8d-f6ad2815f98b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:20.010017 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3346c0ce-a028-4630-bc8d-f6ad2815f98b None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:20.011112 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3346c0ce-a028-4630-bc8d-f6ad2815f98b None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:20.011427 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3346c0ce-a028-4630-bc8d-f6ad2815f98b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:20.013475 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3346c0ce-a028-4630-bc8d-f6ad2815f98b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:20.025651 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 302/610] 10.222.0.50 () {56 vars in 1103 bytes} [Fri May 27 15:01:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:01:20.033545 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3346c0ce-a028-4630-bc8d-f6ad2815f98b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:20.057794 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808b6c8f-1213-4d3b-84d4-a777afce0a2f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:20.122874 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38ba644-27e9-4e57-a967-db5894c001ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:20.123458 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ba644-27e9-4e57-a967-db5894c001ed None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:20.126872 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ba644-27e9-4e57-a967-db5894c001ed None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:20.126872 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38ba644-27e9-4e57-a967-db5894c001ed None admin] PATH_INFO: `/v3/projects/daf12585428b41a2b45bdbc85413a6c4/users/f874edc08438434189f1d14d79291487/roles/13f37584013544e7ad1e2c8f350da6d2` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:20.162881 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38ba644-27e9-4e57-a967-db5894c001ed None admin] RBAC: Authorizing `identity:create_grant(project_id=daf12585428b41a2b45bdbc85413a6c4, user_id=f874edc08438434189f1d14d79291487, role_id=13f37584013544e7ad1e2c8f350da6d2)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:20.162881 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38ba644-27e9-4e57-a967-db5894c001ed None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:20.197702 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 309/611] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 15:01:19 2022] PUT /identity/v3/projects/daf12585428b41a2b45bdbc85413a6c4/users/f874edc08438434189f1d14d79291487/roles/13f37584013544e7ad1e2c8f350da6d2 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:01:20.209665 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:20.218437 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3346c0ce-a028-4630-bc8d-f6ad2815f98b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:20.270586 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9083548-7d84-4369-bc3a-a8e680009075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:20.271426 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9083548-7d84-4369-bc3a-a8e680009075 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:20.271715 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9083548-7d84-4369-bc3a-a8e680009075 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:20.271991 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9083548-7d84-4369-bc3a-a8e680009075 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:20.284113 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9083548-7d84-4369-bc3a-a8e680009075 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:20.285621 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9083548-7d84-4369-bc3a-a8e680009075 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:20.317712 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-f9083548-7d84-4369-bc3a-a8e680009075 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:20.317712 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:20.317712 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:20.317712 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:20.317712 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:20.317712 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:20.317712 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:20.317712 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:20.317712 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:20.317712 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:20.317712 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:20.317712 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:20.317712 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:20.317712 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:20.317712 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:20.319992 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:20.319992 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:20.319992 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:20.319992 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:20.319992 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:20.319992 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:20.319992 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:20.319992 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:20.319992 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:20.319992 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:20.319992 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:20.319992 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:20.319992 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:20.319992 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:20.319992 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:20.319992 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:20.319992 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:20.322124 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:20.322124 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:20.322124 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:20.322124 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:20.322124 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 303/612] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:01:20.324899 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38ba644-27e9-4e57-a967-db5894c001ed None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:20.348423 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:20.446060 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38ba644-27e9-4e57-a967-db5894c001ed None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:20.486372 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9391ee2e-4e26-47c9-8932-80c62e7146b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:20.487082 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391ee2e-4e26-47c9-8932-80c62e7146b1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:20.487373 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391ee2e-4e26-47c9-8932-80c62e7146b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:20.487704 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391ee2e-4e26-47c9-8932-80c62e7146b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:20.488429 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9391ee2e-4e26-47c9-8932-80c62e7146b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:20.488786 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9391ee2e-4e26-47c9-8932-80c62e7146b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:20.490156 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9391ee2e-4e26-47c9-8932-80c62e7146b1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:20.503935 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 310/613] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:20.509010 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9083548-7d84-4369-bc3a-a8e680009075 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:20.622799 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9083548-7d84-4369-bc3a-a8e680009075 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:20.662589 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4eadf1-f444-469e-a597-f2d1c080dc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:20.663421 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4eadf1-f444-469e-a597-f2d1c080dc2a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:20.663703 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4eadf1-f444-469e-a597-f2d1c080dc2a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:20.663983 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4eadf1-f444-469e-a597-f2d1c080dc2a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:20.672234 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4eadf1-f444-469e-a597-f2d1c080dc2a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:20.673726 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4eadf1-f444-469e-a597-f2d1c080dc2a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:20.690697 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-fd4eadf1-f444-469e-a597-f2d1c080dc2a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 15:01:20.690697 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:20.690697 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:20.690697 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:20.690697 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:20.690697 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:20.690697 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:20.690697 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:20.690697 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:20.690697 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:20.690697 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:20.690697 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:20.690697 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 15:01:20.690697 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 15:01:20.690697 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 15:01:20.690697 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 15:01:20.693027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:20.693027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:20.693027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 15:01:20.693027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:20.693027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 15:01:20.693027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:20.693027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:20.693027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:20.693027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:20.693027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:20.693027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:20.693027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:20.693027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:20.693027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:20.693027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:20.693027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:20.693027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:20.693027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:20.695298 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:20.695298 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:20.695298 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 15:01:20.695298 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 15:01:20.695298 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 15:01:20.695298 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 15:01:20.695298 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 15:01:20.695298 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 15:01:20.695298 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 15:01:20.695298 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:20.695298 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 304/614] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 15:01:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:20.699950 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9391ee2e-4e26-47c9-8932-80c62e7146b1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:20.822755 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:20.851596 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9391ee2e-4e26-47c9-8932-80c62e7146b1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:20.899668 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91576b91-4399-4b87-90b6-fad912258af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:20.900259 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91576b91-4399-4b87-90b6-fad912258af0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:20.900546 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91576b91-4399-4b87-90b6-fad912258af0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:20.900827 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91576b91-4399-4b87-90b6-fad912258af0 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:20.901673 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91576b91-4399-4b87-90b6-fad912258af0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:20.901982 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91576b91-4399-4b87-90b6-fad912258af0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:20.903461 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91576b91-4399-4b87-90b6-fad912258af0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:20.926142 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 311/615] 10.222.0.50 () {56 vars in 1135 bytes} [Fri May 27 15:01:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:20.931686 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4eadf1-f444-469e-a597-f2d1c080dc2a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:20.956779 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:21.075102 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4eadf1-f444-469e-a597-f2d1c080dc2a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:21.119094 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dbf1fa-2e86-4e05-8af2-45eb091a10ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:21.120277 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dbf1fa-2e86-4e05-8af2-45eb091a10ba None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:21.120778 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dbf1fa-2e86-4e05-8af2-45eb091a10ba None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:21.121260 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dbf1fa-2e86-4e05-8af2-45eb091a10ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:21.133950 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dbf1fa-2e86-4e05-8af2-45eb091a10ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:21.135743 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dbf1fa-2e86-4e05-8af2-45eb091a10ba None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:21.162899 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-98dbf1fa-2e86-4e05-8af2-45eb091a10ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:21.162899 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:21.162899 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:21.162899 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:21.162899 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:21.162899 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:21.162899 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:21.162899 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:21.162899 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:21.162899 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:21.162899 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:21.162899 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:21.162899 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:21.162899 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:21.162899 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:21.163818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:21.163818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:21.163818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:21.163818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:21.163818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:21.163818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:21.163818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:21.163818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:21.163818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:21.163818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:21.163818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:21.163818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:21.163818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:21.163818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:21.163818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:21.163818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:21.163818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:21.164747 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:21.164747 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:21.164747 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:21.164747 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:21.164747 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 305/616] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:21.165035 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91576b91-4399-4b87-90b6-fad912258af0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:21.342554 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91576b91-4399-4b87-90b6-fad912258af0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:21.392971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cd1f56-d7ac-44dd-97c1-bab5f009748f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:21.393646 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd1f56-d7ac-44dd-97c1-bab5f009748f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:21.393928 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd1f56-d7ac-44dd-97c1-bab5f009748f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:21.394198 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd1f56-d7ac-44dd-97c1-bab5f009748f None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:21.394947 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cd1f56-d7ac-44dd-97c1-bab5f009748f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:21.395249 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cd1f56-d7ac-44dd-97c1-bab5f009748f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:21.396598 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cd1f56-d7ac-44dd-97c1-bab5f009748f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:21.400089 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:21.410929 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 312/617] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:21.415985 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dbf1fa-2e86-4e05-8af2-45eb091a10ba None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:21.524587 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:21.549042 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df870cc9-b2e8-4b30-ad6a-4f0f3fe0eafb None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:21.549359 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df870cc9-b2e8-4b30-ad6a-4f0f3fe0eafb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:21.549605 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df870cc9-b2e8-4b30-ad6a-4f0f3fe0eafb None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:21.551121 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 313/618] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 15:01:21.558817 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c01880-a036-4034-93e0-4f30a1e76e26 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:21.559300 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c01880-a036-4034-93e0-4f30a1e76e26 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:21.559613 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c01880-a036-4034-93e0-4f30a1e76e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:21.578619 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dbf1fa-2e86-4e05-8af2-45eb091a10ba None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:21.591223 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-13c01880-a036-4034-93e0-4f30a1e76e26 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:21.592417 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-13c01880-a036-4034-93e0-4f30a1e76e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:21.623783 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c6504-3b28-4f2c-a5ac-dc9473ab9897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:21.624612 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c6504-3b28-4f2c-a5ac-dc9473ab9897 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:21.624981 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c6504-3b28-4f2c-a5ac-dc9473ab9897 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:21.625262 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9c6504-3b28-4f2c-a5ac-dc9473ab9897 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:21.638589 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9c6504-3b28-4f2c-a5ac-dc9473ab9897 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:21.640317 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9c6504-3b28-4f2c-a5ac-dc9473ab9897 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:21.664982 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-ae9c6504-3b28-4f2c-a5ac-dc9473ab9897 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:21.664982 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:21.664982 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:21.664982 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:21.664982 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:21.664982 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:21.664982 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:21.664982 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:21.664982 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:21.664982 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:21.664982 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:21.664982 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:21.664982 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 15:01:21.664982 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 15:01:21.664982 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 15:01:21.665950 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 15:01:21.665950 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:21.665950 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:21.665950 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:21.665950 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:21.665950 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:21.665950 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:21.665950 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:21.665950 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:21.665950 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:21.665950 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:21.665950 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:21.665950 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:21.665950 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:21.665950 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:21.665950 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:21.665950 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:21.667474 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 15:01:21.667474 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 15:01:21.667474 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 15:01:21.667474 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 15:01:21.667474 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 15:01:21.667474 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 15:01:21.667474 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:21.667474 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:21.667474 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 306/619] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:01:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:21.676372 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c6504-3b28-4f2c-a5ac-dc9473ab9897 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:21.764574 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 314/620] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:01:21.805358 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9c6504-3b28-4f2c-a5ac-dc9473ab9897 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:21.835650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:21.838238 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c01880-a036-4034-93e0-4f30a1e76e26 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:21.865904 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2196b0-56a6-4840-87a1-0a51f5189b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:21.867012 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2196b0-56a6-4840-87a1-0a51f5189b29 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:21.867525 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2196b0-56a6-4840-87a1-0a51f5189b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:21.868020 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2196b0-56a6-4840-87a1-0a51f5189b29 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:21.869014 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2196b0-56a6-4840-87a1-0a51f5189b29 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:21.869524 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2196b0-56a6-4840-87a1-0a51f5189b29 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:21.871204 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2196b0-56a6-4840-87a1-0a51f5189b29 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:21.888074 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 307/621] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 15:01:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:21.894496 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2196b0-56a6-4840-87a1-0a51f5189b29 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:22.000624 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c01880-a036-4034-93e0-4f30a1e76e26 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:22.018655 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2196b0-56a6-4840-87a1-0a51f5189b29 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:22.038048 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:22.050841 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137768cc-df03-4344-b419-e75c5f2e7690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:22.052623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137768cc-df03-4344-b419-e75c5f2e7690 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:22.053381 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137768cc-df03-4344-b419-e75c5f2e7690 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:22.053861 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137768cc-df03-4344-b419-e75c5f2e7690 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:22.057123 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38056246-3a36-47a0-8906-46a5c87c8249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:22.058011 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38056246-3a36-47a0-8906-46a5c87c8249 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:22.058294 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38056246-3a36-47a0-8906-46a5c87c8249 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:22.058572 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38056246-3a36-47a0-8906-46a5c87c8249 None admin] PATH_INFO: `/v3/projects/daf12585428b41a2b45bdbc85413a6c4/users/18903f2eab7c415096e075d47f0b9a4d/roles/13f37584013544e7ad1e2c8f350da6d2` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:22.067004 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137768cc-df03-4344-b419-e75c5f2e7690 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:01:22.074237 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38056246-3a36-47a0-8906-46a5c87c8249 None admin] RBAC: Authorizing `identity:create_grant(project_id=daf12585428b41a2b45bdbc85413a6c4, user_id=18903f2eab7c415096e075d47f0b9a4d, role_id=13f37584013544e7ad1e2c8f350da6d2)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:22.075837 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38056246-3a36-47a0-8906-46a5c87c8249 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:22.077707 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-137768cc-df03-4344-b419-e75c5f2e7690 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 15:01:22.077707 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:22.077707 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:22.077707 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:22.077707 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:22.077707 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:22.077707 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:22.077707 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:22.077707 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:22.077707 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:22.077707 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:22.077707 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:22.077707 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:01:22.077707 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:01:22.077707 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:01:22.077707 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:01:22.078673 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:01:22.078673 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:01:22.078673 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:01:22.078673 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:01:22.078673 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:22.078673 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:22.078673 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:22.078673 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:22.078673 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:22.078673 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:22.078673 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:22.078673 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:22.078673 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:22.078673 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:22.078673 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:22.078673 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:22.079554 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:22.079554 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:22.079554 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:22.079554 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:22.079554 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:01:22.079554 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:01:22.079554 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:01:22.079554 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:01:22.079554 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:01:22.079554 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:01:22.079554 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 15:01:22.079554 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:22.080850 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 315/622] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 15:01:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:22.082749 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e67ac-e19a-47c7-84df-4047bdb2076c None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:22.083062 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e67ac-e19a-47c7-84df-4047bdb2076c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:22.083344 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e67ac-e19a-47c7-84df-4047bdb2076c None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:22.085558 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 316/623] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:22.087613 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e67ac-e19a-47c7-84df-4047bdb2076c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:22.099087 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 308/624] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 15:01:21 2022] PUT /identity/v3/projects/daf12585428b41a2b45bdbc85413a6c4/users/18903f2eab7c415096e075d47f0b9a4d/roles/13f37584013544e7ad1e2c8f350da6d2 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:01:22.100495 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5356570-e122-4f2f-b0af-baabae5396db None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:22.100797 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5356570-e122-4f2f-b0af-baabae5396db None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:22.101061 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5356570-e122-4f2f-b0af-baabae5396db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:22.120368 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e5356570-e122-4f2f-b0af-baabae5396db None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:22.121620 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e5356570-e122-4f2f-b0af-baabae5396db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:22.164588 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:22.239381 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e67ac-e19a-47c7-84df-4047bdb2076c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:22.282937 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6f4075-1452-4d78-aa04-584c9db385c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:22.284023 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6f4075-1452-4d78-aa04-584c9db385c6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:22.284992 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6f4075-1452-4d78-aa04-584c9db385c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:22.285507 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6f4075-1452-4d78-aa04-584c9db385c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:22.286471 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6f4075-1452-4d78-aa04-584c9db385c6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:22.287028 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6f4075-1452-4d78-aa04-584c9db385c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:22.288905 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6f4075-1452-4d78-aa04-584c9db385c6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:22.299589 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 317/625] 10.222.0.50 () {56 vars in 1103 bytes} [Fri May 27 15:01:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:22.304735 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6f4075-1452-4d78-aa04-584c9db385c6 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:22.316653 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 309/626] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:01:22.356693 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5356570-e122-4f2f-b0af-baabae5396db None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:22.363371 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:22.433013 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6f4075-1452-4d78-aa04-584c9db385c6 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:22.476193 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5356570-e122-4f2f-b0af-baabae5396db None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:22.476999 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac0c8d9-6b88-4205-8921-8afb6a707c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:22.478041 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac0c8d9-6b88-4205-8921-8afb6a707c34 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:22.478351 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac0c8d9-6b88-4205-8921-8afb6a707c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:22.479250 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac0c8d9-6b88-4205-8921-8afb6a707c34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:22.500295 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac0c8d9-6b88-4205-8921-8afb6a707c34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:22.501903 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac0c8d9-6b88-4205-8921-8afb6a707c34 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:22.516681 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06db1063-413f-4b77-84c5-52acfb96777e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:22.517712 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db1063-413f-4b77-84c5-52acfb96777e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:22.518139 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db1063-413f-4b77-84c5-52acfb96777e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:22.518569 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06db1063-413f-4b77-84c5-52acfb96777e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:22.520662 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-cac0c8d9-6b88-4205-8921-8afb6a707c34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:22.520662 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:22.520662 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:22.520662 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:22.520662 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:22.520662 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:22.520662 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:22.520662 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:22.520662 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:22.520662 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:22.520662 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:22.520662 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:22.520662 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:22.520662 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:22.520662 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:22.521711 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:22.521711 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:22.521711 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:22.521711 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:22.521711 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:22.521711 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:22.521711 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:22.521711 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:22.521711 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:22.521711 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:22.521711 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:22.521711 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:22.521711 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:22.521711 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:22.521711 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:22.521711 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:22.521711 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:22.522802 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:22.522802 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:22.522802 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:22.522802 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:22.524618 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 318/627] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:22.533952 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58e9d5-1344-445c-ae89-defc7ad56be7 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:22.533952 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58e9d5-1344-445c-ae89-defc7ad56be7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:22.533952 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58e9d5-1344-445c-ae89-defc7ad56be7 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:22.533952 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 319/628] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:22.533952 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d58e9d5-1344-445c-ae89-defc7ad56be7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:22.543371 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06db1063-413f-4b77-84c5-52acfb96777e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:01:22.553350 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-06db1063-413f-4b77-84c5-52acfb96777e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 15:01:22.553350 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:22.553350 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:22.553350 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:22.553350 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:22.553350 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:22.553350 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:22.553350 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:22.553350 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:22.553350 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:22.553350 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:22.553350 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:22.553350 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:01:22.553350 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:01:22.553350 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:01:22.553350 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:01:22.554476 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:01:22.554476 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:01:22.554476 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:01:22.554476 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:01:22.554476 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:22.554476 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:22.554476 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:22.554476 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:22.554476 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:22.554476 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:22.554476 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:22.554476 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:22.554476 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:22.554476 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:22.554476 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:22.554476 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:22.555614 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:22.555614 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:22.555614 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:22.555614 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:22.555614 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:01:22.555614 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:01:22.555614 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:01:22.555614 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:01:22.555614 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:01:22.555614 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:01:22.555614 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 15:01:22.555614 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:22.558439 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 310/629] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 15:01:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:01:22.565871 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231173e7-6def-4618-bc01-4cc76499f639 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:22.567158 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231173e7-6def-4618-bc01-4cc76499f639 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:22.567593 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231173e7-6def-4618-bc01-4cc76499f639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:22.586486 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-231173e7-6def-4618-bc01-4cc76499f639 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:22.588269 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-231173e7-6def-4618-bc01-4cc76499f639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:22.604656 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:22.645612 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:22.650487 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d58e9d5-1344-445c-ae89-defc7ad56be7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:22.671610 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 311/630] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:01:22.672827 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231173e7-6def-4618-bc01-4cc76499f639 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:22.688279 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeb0bde-daa9-477a-8ce1-211432c4ecf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:22.689316 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb0bde-daa9-477a-8ce1-211432c4ecf2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:22.689698 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb0bde-daa9-477a-8ce1-211432c4ecf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:22.690329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeb0bde-daa9-477a-8ce1-211432c4ecf2 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:22.699089 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeb0bde-daa9-477a-8ce1-211432c4ecf2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:22.699619 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeb0bde-daa9-477a-8ce1-211432c4ecf2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:22.701177 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeb0bde-daa9-477a-8ce1-211432c4ecf2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:22.719233 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 320/631] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:22.719233 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeb0bde-daa9-477a-8ce1-211432c4ecf2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:22.795503 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231173e7-6def-4618-bc01-4cc76499f639 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:22.832826 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b515b628-d054-467d-baba-df5ae1d03ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:22.833780 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b515b628-d054-467d-baba-df5ae1d03ed3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:22.834274 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b515b628-d054-467d-baba-df5ae1d03ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:22.834689 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b515b628-d054-467d-baba-df5ae1d03ed3 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:22.835614 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b515b628-d054-467d-baba-df5ae1d03ed3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:22.836077 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b515b628-d054-467d-baba-df5ae1d03ed3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:22.837570 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b515b628-d054-467d-baba-df5ae1d03ed3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:22.838325 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeb0bde-daa9-477a-8ce1-211432c4ecf2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:22.852164 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 312/632] 10.222.0.50 () {56 vars in 1103 bytes} [Fri May 27 15:01:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:22.853501 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b515b628-d054-467d-baba-df5ae1d03ed3 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:22.871871 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:22.890941 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8903f9fc-e7e9-4c83-a3e4-2afefc70d528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:22.891975 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903f9fc-e7e9-4c83-a3e4-2afefc70d528 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:22.892343 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903f9fc-e7e9-4c83-a3e4-2afefc70d528 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:22.892672 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8903f9fc-e7e9-4c83-a3e4-2afefc70d528 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:22.906729 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8903f9fc-e7e9-4c83-a3e4-2afefc70d528 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:01:22.917666 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-8903f9fc-e7e9-4c83-a3e4-2afefc70d528 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 15:01:22.917666 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:22.917666 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:22.917666 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:22.917666 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:22.917666 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:22.917666 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:22.917666 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:22.917666 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:22.917666 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:22.917666 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:22.917666 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:22.917666 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:01:22.917666 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:01:22.917666 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:01:22.917666 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:01:22.918828 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:01:22.918828 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:01:22.918828 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:01:22.918828 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:01:22.918828 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:22.918828 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:22.918828 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:22.918828 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:22.918828 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:22.918828 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:22.918828 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:22.918828 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:22.918828 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:22.918828 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:22.918828 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:22.918828 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:22.919896 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:22.919896 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:22.919896 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:22.919896 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:22.919896 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:01:22.919896 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:01:22.919896 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:01:22.919896 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:01:22.919896 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:01:22.919896 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:01:22.919896 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 15:01:22.919896 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:22.922922 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 321/633] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 15:01:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:22.923957 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8903f9fc-e7e9-4c83-a3e4-2afefc70d528 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:22.984455 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:22.988740 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b515b628-d054-467d-baba-df5ae1d03ed3 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:23.032863 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8903f9fc-e7e9-4c83-a3e4-2afefc70d528 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:23.038992 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38afcebe-57f0-456a-9752-068a9d07cd9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:23.039795 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38afcebe-57f0-456a-9752-068a9d07cd9e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:23.040082 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38afcebe-57f0-456a-9752-068a9d07cd9e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:23.040363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38afcebe-57f0-456a-9752-068a9d07cd9e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:23.051936 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38afcebe-57f0-456a-9752-068a9d07cd9e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:23.054062 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38afcebe-57f0-456a-9752-068a9d07cd9e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:23.070646 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b588b38b-51d0-4ed0-beb4-2d4c442fef0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:23.071467 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b588b38b-51d0-4ed0-beb4-2d4c442fef0e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:23.071746 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b588b38b-51d0-4ed0-beb4-2d4c442fef0e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:23.072015 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b588b38b-51d0-4ed0-beb4-2d4c442fef0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:23.072790 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-38afcebe-57f0-456a-9752-068a9d07cd9e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 15:01:23.072790 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:23.072790 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:23.072790 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:23.072790 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:23.072790 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:23.072790 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:23.072790 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:23.072790 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:23.072790 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:23.072790 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:23.072790 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:23.072790 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 15:01:23.072790 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 15:01:23.072790 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 15:01:23.072790 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 15:01:23.073706 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:23.073706 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:23.073706 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 15:01:23.073706 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:23.073706 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 15:01:23.073706 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:23.073706 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:23.073706 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:23.073706 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:23.073706 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:23.073706 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:23.073706 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:23.073706 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:23.073706 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:23.073706 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:23.073706 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:23.073706 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:23.073706 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:23.074680 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:23.074680 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:23.074680 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 15:01:23.074680 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 15:01:23.074680 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 15:01:23.074680 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 15:01:23.074680 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 15:01:23.074680 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 15:01:23.074680 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 15:01:23.074680 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:23.074680 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 313/634] 10.222.0.50 () {56 vars in 1093 bytes} [Fri May 27 15:01:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:23.075753 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38afcebe-57f0-456a-9752-068a9d07cd9e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:23.089072 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b588b38b-51d0-4ed0-beb4-2d4c442fef0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:23.090525 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b588b38b-51d0-4ed0-beb4-2d4c442fef0e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:23.109518 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-b588b38b-51d0-4ed0-beb4-2d4c442fef0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:23.109518 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:23.109518 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:23.109518 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:23.109518 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:23.109518 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:23.109518 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:23.109518 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:23.109518 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:23.109518 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:23.109518 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:23.109518 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:23.109518 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:23.109518 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:23.109518 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:23.110451 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:23.110451 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:23.110451 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:23.110451 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:23.110451 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:23.110451 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:23.110451 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:23.110451 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:23.110451 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:23.110451 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:23.110451 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:23.110451 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:23.110451 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:23.110451 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:23.110451 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:23.110451 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:23.110451 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:23.111393 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:23.111393 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:23.111393 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:23.111393 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:23.115140 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 322/635] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:23.116287 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a4a6ea-7a4c-4089-b01e-0926b1ae67eb None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:23.116574 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a4a6ea-7a4c-4089-b01e-0926b1ae67eb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:23.116847 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a4a6ea-7a4c-4089-b01e-0926b1ae67eb None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:23.122163 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 323/636] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:23.122163 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a4a6ea-7a4c-4089-b01e-0926b1ae67eb None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:23.160715 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:23.193112 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38afcebe-57f0-456a-9752-068a9d07cd9e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:23.198841 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:23.228590 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d94f1b-dc51-4061-a430-0969b82970ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:23.229504 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d94f1b-dc51-4061-a430-0969b82970ac None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:23.229951 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d94f1b-dc51-4061-a430-0969b82970ac None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:23.230230 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d94f1b-dc51-4061-a430-0969b82970ac None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:23.231000 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d94f1b-dc51-4061-a430-0969b82970ac None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:23.231364 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d94f1b-dc51-4061-a430-0969b82970ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:23.232769 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d94f1b-dc51-4061-a430-0969b82970ac None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:23.233100 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a4a6ea-7a4c-4089-b01e-0926b1ae67eb None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:23.241706 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 314/637] 10.222.0.50 () {56 vars in 1103 bytes} [Fri May 27 15:01:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:23.243741 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d94f1b-dc51-4061-a430-0969b82970ac None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:23.268902 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5b10b5-fd0d-4071-9f1c-a10bb40fa55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:23.269571 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5b10b5-fd0d-4071-9f1c-a10bb40fa55f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:23.269850 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5b10b5-fd0d-4071-9f1c-a10bb40fa55f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:23.270119 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5b10b5-fd0d-4071-9f1c-a10bb40fa55f None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:23.270971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5b10b5-fd0d-4071-9f1c-a10bb40fa55f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:23.271279 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5b10b5-fd0d-4071-9f1c-a10bb40fa55f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:23.272685 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5b10b5-fd0d-4071-9f1c-a10bb40fa55f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:23.295490 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 324/638] 10.222.0.50 () {56 vars in 1133 bytes} [Fri May 27 15:01:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:23.295490 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed7c97-284a-4b52-a544-15a5abe99c70 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:23.295490 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed7c97-284a-4b52-a544-15a5abe99c70 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:23.295490 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed7c97-284a-4b52-a544-15a5abe99c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:23.311942 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-64ed7c97-284a-4b52-a544-15a5abe99c70 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:23.313020 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-64ed7c97-284a-4b52-a544-15a5abe99c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:23.353565 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d94f1b-dc51-4061-a430-0969b82970ac None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:23.381835 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 325/639] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:01:23.382511 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed7c97-284a-4b52-a544-15a5abe99c70 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:23.396892 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74142ffd-13ad-4ac3-b9be-6fda32df772c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:23.398198 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74142ffd-13ad-4ac3-b9be-6fda32df772c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:23.398708 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74142ffd-13ad-4ac3-b9be-6fda32df772c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:23.399258 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74142ffd-13ad-4ac3-b9be-6fda32df772c None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:23.400210 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74142ffd-13ad-4ac3-b9be-6fda32df772c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:23.400726 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74142ffd-13ad-4ac3-b9be-6fda32df772c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:23.402516 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74142ffd-13ad-4ac3-b9be-6fda32df772c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:23.404626 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:23.422125 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 315/640] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:23.422125 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74142ffd-13ad-4ac3-b9be-6fda32df772c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:23.453891 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:23.504924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed7c97-284a-4b52-a544-15a5abe99c70 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:23.532021 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74142ffd-13ad-4ac3-b9be-6fda32df772c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:23.542570 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d7a31-c168-4d24-8d70-cdf0fdddb3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:23.543393 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d7a31-c168-4d24-8d70-cdf0fdddb3a8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:23.543673 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d7a31-c168-4d24-8d70-cdf0fdddb3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:23.543945 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d7a31-c168-4d24-8d70-cdf0fdddb3a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:23.554979 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15d7a31-c168-4d24-8d70-cdf0fdddb3a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:23.556414 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15d7a31-c168-4d24-8d70-cdf0fdddb3a8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:23.566927 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a436a0c-27d0-4d72-ae4c-985fef3aecc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:23.567533 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a436a0c-27d0-4d72-ae4c-985fef3aecc4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:23.567817 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a436a0c-27d0-4d72-ae4c-985fef3aecc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:23.568092 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a436a0c-27d0-4d72-ae4c-985fef3aecc4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:23.574748 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-c15d7a31-c168-4d24-8d70-cdf0fdddb3a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:23.574748 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:23.574748 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:23.574748 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:23.574748 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:23.574748 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:23.574748 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:23.574748 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:23.574748 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:23.574748 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:23.574748 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:23.574748 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:23.574748 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:23.574748 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:23.574748 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:23.575665 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:23.575665 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:23.575665 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:23.575665 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:23.575665 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:23.575665 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:23.575665 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:23.575665 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:23.575665 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:23.575665 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:23.575665 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:23.575665 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:23.575665 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:23.575665 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:23.575665 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:23.575665 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:23.575665 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:23.581770 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:23.581770 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:23.581770 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:23.581770 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:23.581770 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 326/641] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:23.581770 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d7a31-c168-4d24-8d70-cdf0fdddb3a8 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:23.631859 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:23.641626 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a436a0c-27d0-4d72-ae4c-985fef3aecc4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:23.643395 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a436a0c-27d0-4d72-ae4c-985fef3aecc4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:23.688288 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d7a31-c168-4d24-8d70-cdf0fdddb3a8 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:23.703439 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:23.728703 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96164b63-9035-48cc-914d-c427882b605c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:23.729500 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96164b63-9035-48cc-914d-c427882b605c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:23.729830 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96164b63-9035-48cc-914d-c427882b605c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:23.730108 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96164b63-9035-48cc-914d-c427882b605c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:23.747305 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 316/642] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:01:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 15:01:23.747451 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96164b63-9035-48cc-914d-c427882b605c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:01:23.747530 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a436a0c-27d0-4d72-ae4c-985fef3aecc4 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:23.754548 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-96164b63-9035-48cc-914d-c427882b605c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 15:01:23.754548 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:23.754548 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:23.754548 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:23.754548 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:23.754548 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:23.754548 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:23.754548 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:23.754548 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:23.754548 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:23.754548 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:23.754548 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:23.754548 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:01:23.754548 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:01:23.754548 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:01:23.754548 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:01:23.757470 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:01:23.757470 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:01:23.757470 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:01:23.757470 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:01:23.757470 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:23.757470 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:23.757470 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:23.757470 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:23.757470 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:23.757470 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:23.757470 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:23.757470 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:23.757470 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:23.757470 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:23.757470 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:23.757470 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:23.758328 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:23.758328 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:23.758328 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:23.758328 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:23.758328 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:01:23.758328 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:01:23.758328 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:01:23.758328 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:01:23.758328 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:01:23.758328 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:01:23.758328 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 15:01:23.758328 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:23.758328 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 327/643] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 15:01:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:23.758328 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96164b63-9035-48cc-914d-c427882b605c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:23.851670 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:23.859088 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a436a0c-27d0-4d72-ae4c-985fef3aecc4 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:23.871938 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96164b63-9035-48cc-914d-c427882b605c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:23.894116 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102ee647-32f6-4782-8430-7eae1cba8301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:23.894942 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ee647-32f6-4782-8430-7eae1cba8301 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:23.895225 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ee647-32f6-4782-8430-7eae1cba8301 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:23.895500 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102ee647-32f6-4782-8430-7eae1cba8301 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:23.902729 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102ee647-32f6-4782-8430-7eae1cba8301 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:23.904178 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102ee647-32f6-4782-8430-7eae1cba8301 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:23.909724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f449bc-4ff4-4643-beb6-135c10239862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:23.910577 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f449bc-4ff4-4643-beb6-135c10239862 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:23.910885 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f449bc-4ff4-4643-beb6-135c10239862 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:23.911163 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f449bc-4ff4-4643-beb6-135c10239862 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:23.911881 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f449bc-4ff4-4643-beb6-135c10239862 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:23.912187 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f449bc-4ff4-4643-beb6-135c10239862 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:23.913603 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f449bc-4ff4-4643-beb6-135c10239862 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:23.918953 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-102ee647-32f6-4782-8430-7eae1cba8301 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 27 15:01:23.918953 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:23.918953 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:23.918953 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:23.918953 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:23.918953 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:23.918953 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:23.918953 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:23.918953 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:23.918953 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:23.918953 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:23.918953 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:23.918953 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 15:01:23.918953 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 15:01:23.918953 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 15:01:23.918953 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 15:01:23.919868 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:23.919868 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:23.919868 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 15:01:23.919868 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:23.919868 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 15:01:23.919868 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:23.919868 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:23.919868 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:23.919868 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:23.919868 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:23.919868 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:23.919868 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:23.919868 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:23.919868 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:23.919868 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:23.919868 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:23.919868 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:23.919868 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:23.925622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:23.925622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:23.925622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 15:01:23.925622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 15:01:23.925622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 15:01:23.925622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 15:01:23.925622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 15:01:23.925622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 15:01:23.925622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 27 15:01:23.925622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:23.925622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 317/644] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:01:23 2022] GET /identity/v3/users/glance => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:23.925622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102ee647-32f6-4782-8430-7eae1cba8301 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:23.937648 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 328/645] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:23.938689 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f449bc-4ff4-4643-beb6-135c10239862 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:23.971952 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:24.037605 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102ee647-32f6-4782-8430-7eae1cba8301 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:24.054229 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f449bc-4ff4-4643-beb6-135c10239862 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:24.074822 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1788ba3-ff38-4bf2-8abd-52b48291ae8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:24.076178 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1788ba3-ff38-4bf2-8abd-52b48291ae8d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:24.076559 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1788ba3-ff38-4bf2-8abd-52b48291ae8d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:24.076951 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1788ba3-ff38-4bf2-8abd-52b48291ae8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:24.087856 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1788ba3-ff38-4bf2-8abd-52b48291ae8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:24.089726 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1788ba3-ff38-4bf2-8abd-52b48291ae8d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:24.091189 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0f46c3-3e7a-4dfc-a4ed-cc684f11956d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:24.092011 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0f46c3-3e7a-4dfc-a4ed-cc684f11956d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:24.092609 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0f46c3-3e7a-4dfc-a4ed-cc684f11956d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:24.092928 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0f46c3-3e7a-4dfc-a4ed-cc684f11956d None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:24.093679 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0f46c3-3e7a-4dfc-a4ed-cc684f11956d None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:24.093984 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0f46c3-3e7a-4dfc-a4ed-cc684f11956d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:24.095356 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0f46c3-3e7a-4dfc-a4ed-cc684f11956d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:24.104738 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 329/646] 10.222.0.50 () {56 vars in 1103 bytes} [Fri May 27 15:01:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:24.105304 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0f46c3-3e7a-4dfc-a4ed-cc684f11956d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:24.112405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-c1788ba3-ff38-4bf2-8abd-52b48291ae8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:24.112405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:24.112405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:24.112405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:24.112405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:24.112405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:24.112405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:24.112405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:24.112405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:24.112405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:24.112405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:24.112405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:24.112405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:24.112405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:24.112405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:24.113470 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:24.113470 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:24.113470 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:24.113470 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:24.113470 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:24.113470 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:24.113470 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:24.113470 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:24.113470 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:24.113470 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:24.113470 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:24.113470 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:24.113470 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:24.113470 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:24.113470 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:24.113470 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:24.113470 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:24.114603 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:24.114603 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:24.114603 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:24.114603 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:24.120663 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 318/647] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:24.120663 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1788ba3-ff38-4bf2-8abd-52b48291ae8d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:24.130488 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:24.219185 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:24.226636 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0f46c3-3e7a-4dfc-a4ed-cc684f11956d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:24.236877 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1788ba3-ff38-4bf2-8abd-52b48291ae8d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:24.263747 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d97788d-c4b8-4a6c-9771-48619c1cf536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:24.264544 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97788d-c4b8-4a6c-9771-48619c1cf536 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:24.264952 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97788d-c4b8-4a6c-9771-48619c1cf536 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:24.265339 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d97788d-c4b8-4a6c-9771-48619c1cf536 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:24.266273 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d97788d-c4b8-4a6c-9771-48619c1cf536 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:24.266669 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d97788d-c4b8-4a6c-9771-48619c1cf536 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:24.268198 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d97788d-c4b8-4a6c-9771-48619c1cf536 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:24.271291 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2b3b04-4ee5-4a51-b185-913bf5c80855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:24.272080 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b3b04-4ee5-4a51-b185-913bf5c80855 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:24.272365 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b3b04-4ee5-4a51-b185-913bf5c80855 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:24.272638 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b3b04-4ee5-4a51-b185-913bf5c80855 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:24.281440 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2b3b04-4ee5-4a51-b185-913bf5c80855 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:24.283430 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2b3b04-4ee5-4a51-b185-913bf5c80855 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:24.286168 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 330/648] 10.222.0.50 () {56 vars in 1137 bytes} [Fri May 27 15:01:24 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:24.289843 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d97788d-c4b8-4a6c-9771-48619c1cf536 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:24.299654 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-ff2b3b04-4ee5-4a51-b185-913bf5c80855 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 27 15:01:24.299654 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:24.299654 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:24.299654 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:24.299654 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:24.299654 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:24.299654 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:24.299654 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:24.299654 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:24.299654 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:24.299654 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:24.299654 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:24.299654 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 15:01:24.299654 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 15:01:24.299654 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 15:01:24.299654 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 15:01:24.300896 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:24.300896 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:24.300896 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 15:01:24.300896 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:24.300896 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 15:01:24.300896 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:24.300896 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:24.300896 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:24.300896 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:24.300896 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:24.300896 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:24.300896 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:24.300896 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:24.300896 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:24.300896 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:24.300896 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:24.300896 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:24.300896 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:24.302077 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:24.302077 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:24.302077 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 15:01:24.302077 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 15:01:24.302077 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 15:01:24.302077 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 15:01:24.302077 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 15:01:24.302077 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 15:01:24.302077 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 27 15:01:24.302077 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:24.308763 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 319/649] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:01:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:24.308763 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2b3b04-4ee5-4a51-b185-913bf5c80855 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:24.374962 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:24.406459 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d97788d-c4b8-4a6c-9771-48619c1cf536 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:24.408669 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2b3b04-4ee5-4a51-b185-913bf5c80855 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:24.443610 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e2c2c-6bb7-4d5b-8f25-5398c3443a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:24.443974 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3a0953-6a6c-493f-8b22-444fae457b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:24.444632 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a0953-6a6c-493f-8b22-444fae457b57 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:24.444783 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e2c2c-6bb7-4d5b-8f25-5398c3443a22 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:24.444911 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a0953-6a6c-493f-8b22-444fae457b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:24.445182 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3a0953-6a6c-493f-8b22-444fae457b57 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:24.445309 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e2c2c-6bb7-4d5b-8f25-5398c3443a22 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:24.445680 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e2c2c-6bb7-4d5b-8f25-5398c3443a22 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:24.445899 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3a0953-6a6c-493f-8b22-444fae457b57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:24.446206 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3a0953-6a6c-493f-8b22-444fae457b57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:24.447590 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3a0953-6a6c-493f-8b22-444fae457b57 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:24.458332 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48e2c2c-6bb7-4d5b-8f25-5398c3443a22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:24.464514 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 320/650] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:24.464654 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48e2c2c-6bb7-4d5b-8f25-5398c3443a22 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:24.464746 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3a0953-6a6c-493f-8b22-444fae457b57 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:24.483170 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-b48e2c2c-6bb7-4d5b-8f25-5398c3443a22 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:24.483170 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:24.483170 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:24.483170 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:24.483170 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:24.483170 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:24.483170 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:24.483170 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:24.483170 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:24.483170 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:24.483170 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:24.483170 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:24.483170 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:24.483170 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:24.483170 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:24.484831 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:24.484831 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:24.484831 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:24.484831 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:24.484831 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:24.484831 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:24.484831 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:24.484831 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:24.484831 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:24.484831 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:24.484831 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:24.484831 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:24.484831 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:24.484831 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:24.484831 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:24.484831 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:24.484831 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:24.485943 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:24.485943 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:24.485943 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:24.485943 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:24.489676 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 331/651] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:24.490526 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e2c2c-6bb7-4d5b-8f25-5398c3443a22 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:24.494216 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:24.567051 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3a0953-6a6c-493f-8b22-444fae457b57 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:24.587738 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e2c2c-6bb7-4d5b-8f25-5398c3443a22 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:24.591302 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:24.600422 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7793f05a-b383-4c5c-bdb7-f6d204c11b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:24.601068 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7793f05a-b383-4c5c-bdb7-f6d204c11b4e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:24.601358 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7793f05a-b383-4c5c-bdb7-f6d204c11b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:24.601638 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7793f05a-b383-4c5c-bdb7-f6d204c11b4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:24.622348 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94be103c-584e-43b7-86d0-e49a0200263a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:24.623052 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be103c-584e-43b7-86d0-e49a0200263a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:24.623331 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be103c-584e-43b7-86d0-e49a0200263a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:24.623607 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be103c-584e-43b7-86d0-e49a0200263a None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:24.624551 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94be103c-584e-43b7-86d0-e49a0200263a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:24.624855 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94be103c-584e-43b7-86d0-e49a0200263a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:24.626329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94be103c-584e-43b7-86d0-e49a0200263a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:24.643023 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 332/652] 10.222.0.50 () {56 vars in 1137 bytes} [Fri May 27 15:01:24 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:24.643905 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94be103c-584e-43b7-86d0-e49a0200263a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:24.668564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7793f05a-b383-4c5c-bdb7-f6d204c11b4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:24.670007 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7793f05a-b383-4c5c-bdb7-f6d204c11b4e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:24.703683 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:24.741050 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94be103c-584e-43b7-86d0-e49a0200263a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:24.766773 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 321/653] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:01:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 15:01:24.767728 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7793f05a-b383-4c5c-bdb7-f6d204c11b4e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:24.779069 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9c481a-6af2-4fe3-81e0-1633f7e126bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:24.779873 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c481a-6af2-4fe3-81e0-1633f7e126bc None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:24.780155 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c481a-6af2-4fe3-81e0-1633f7e126bc None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:24.780539 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9c481a-6af2-4fe3-81e0-1633f7e126bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:24.788940 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9c481a-6af2-4fe3-81e0-1633f7e126bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:24.790386 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9c481a-6af2-4fe3-81e0-1633f7e126bc None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:24.801808 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:24.804274 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-2b9c481a-6af2-4fe3-81e0-1633f7e126bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:24.804274 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:24.804274 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:24.804274 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:24.804274 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:24.804274 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:24.804274 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:24.804274 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:24.804274 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:24.804274 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:24.804274 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:24.804274 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:24.804274 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 15:01:24.804274 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 15:01:24.804274 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 15:01:24.805184 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 15:01:24.805184 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:24.805184 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:24.805184 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:24.805184 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:24.805184 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:24.805184 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:24.805184 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:24.805184 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:24.805184 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:24.805184 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:24.805184 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:24.805184 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:24.805184 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:24.805184 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:24.805184 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:24.805184 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:24.806079 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 15:01:24.806079 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 15:01:24.806079 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 15:01:24.806079 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 15:01:24.806079 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 15:01:24.806079 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 15:01:24.806079 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:24.806079 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:24.810325 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 333/654] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:01:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:24.812117 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829ba87-f416-4503-b944-0e99d16e6a4a None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:24.812398 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829ba87-f416-4503-b944-0e99d16e6a4a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:24.812678 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829ba87-f416-4503-b944-0e99d16e6a4a None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:24.814099 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 334/655] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:24.814728 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8829ba87-f416-4503-b944-0e99d16e6a4a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:24.878053 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7793f05a-b383-4c5c-bdb7-f6d204c11b4e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:24.911980 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017e8184-e9d9-497c-ac63-9dcb5cf3ba41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:24.912639 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e8184-e9d9-497c-ac63-9dcb5cf3ba41 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:24.912921 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e8184-e9d9-497c-ac63-9dcb5cf3ba41 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:24.913195 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e8184-e9d9-497c-ac63-9dcb5cf3ba41 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:24.913905 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017e8184-e9d9-497c-ac63-9dcb5cf3ba41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:24.914027 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8829ba87-f416-4503-b944-0e99d16e6a4a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:24.914454 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017e8184-e9d9-497c-ac63-9dcb5cf3ba41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:24.915597 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017e8184-e9d9-497c-ac63-9dcb5cf3ba41 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:24.927036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 322/656] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:24.927831 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017e8184-e9d9-497c-ac63-9dcb5cf3ba41 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:24.938468 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:24.950402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc3582-84ff-48ce-9bd0-c25581232dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:24.951077 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc3582-84ff-48ce-9bd0-c25581232dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:24.951363 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc3582-84ff-48ce-9bd0-c25581232dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:24.951636 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc3582-84ff-48ce-9bd0-c25581232dd1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:25.012159 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:25.018361 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc3582-84ff-48ce-9bd0-c25581232dd1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:25.019829 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc3582-84ff-48ce-9bd0-c25581232dd1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:25.028106 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017e8184-e9d9-497c-ac63-9dcb5cf3ba41 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:25.060260 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc63806-376e-46c4-a520-ee902899d673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:25.061037 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc63806-376e-46c4-a520-ee902899d673 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:25.061314 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc63806-376e-46c4-a520-ee902899d673 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:25.062260 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc63806-376e-46c4-a520-ee902899d673 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:25.072458 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc63806-376e-46c4-a520-ee902899d673 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:25.073878 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc63806-376e-46c4-a520-ee902899d673 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:25.090405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-efc63806-376e-46c4-a520-ee902899d673 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:25.090405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:25.090405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:25.090405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:25.090405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:25.090405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:25.090405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:25.090405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:25.090405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:25.090405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:25.090405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:25.090405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:25.090405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:25.090405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:25.090405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:25.091749 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:25.091749 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:25.091749 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:25.091749 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:25.091749 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:25.091749 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:25.091749 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:25.091749 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:25.091749 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:25.091749 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:25.091749 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:25.091749 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:25.091749 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:25.091749 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:25.091749 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:25.091749 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:25.091749 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:25.092647 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:25.092647 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:25.092647 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:25.092647 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:25.092647 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 323/657] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:25.092647 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc63806-376e-46c4-a520-ee902899d673 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:25.118104 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 335/658] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:01:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 15:01:25.119490 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b3e43-9b7f-48c2-b0ae-a14e437925a1 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:25.119781 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b3e43-9b7f-48c2-b0ae-a14e437925a1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:25.120056 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476b3e43-9b7f-48c2-b0ae-a14e437925a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:25.139781 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-476b3e43-9b7f-48c2-b0ae-a14e437925a1 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:25.141169 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-476b3e43-9b7f-48c2-b0ae-a14e437925a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:25.175661 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:25.209942 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc63806-376e-46c4-a520-ee902899d673 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:25.225828 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 336/659] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:01:25.226665 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b3e43-9b7f-48c2-b0ae-a14e437925a1 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:25.243975 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8895ec-9f6a-487a-83bf-42a8ca9378bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:25.244737 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8895ec-9f6a-487a-83bf-42a8ca9378bb None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:25.245016 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8895ec-9f6a-487a-83bf-42a8ca9378bb None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:25.245283 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8895ec-9f6a-487a-83bf-42a8ca9378bb None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:25.246052 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8895ec-9f6a-487a-83bf-42a8ca9378bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:25.246358 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8895ec-9f6a-487a-83bf-42a8ca9378bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:25.247907 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8895ec-9f6a-487a-83bf-42a8ca9378bb None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:25.255630 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:25.262002 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 324/660] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 15:01:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:25.262793 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8895ec-9f6a-487a-83bf-42a8ca9378bb None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:25.324719 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476b3e43-9b7f-48c2-b0ae-a14e437925a1 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:25.358060 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac0567-7970-4bb9-9325-e6abbd07542b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:25.359213 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8895ec-9f6a-487a-83bf-42a8ca9378bb None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:25.359602 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac0567-7970-4bb9-9325-e6abbd07542b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:25.359894 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac0567-7970-4bb9-9325-e6abbd07542b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:25.360171 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac0567-7970-4bb9-9325-e6abbd07542b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:25.367069 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feac0567-7970-4bb9-9325-e6abbd07542b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:25.368493 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feac0567-7970-4bb9-9325-e6abbd07542b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:25.382715 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-feac0567-7970-4bb9-9325-e6abbd07542b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 27 15:01:25.382715 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:25.382715 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:25.382715 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:25.382715 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:25.382715 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:25.382715 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:25.382715 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:25.382715 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:25.382715 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:25.382715 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:25.382715 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:25.382715 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 15:01:25.382715 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 15:01:25.382715 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 15:01:25.382715 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 15:01:25.383683 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:25.383683 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:25.383683 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 15:01:25.383683 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:25.383683 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 15:01:25.383683 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:25.383683 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:25.383683 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:25.383683 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:25.383683 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:25.383683 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:25.383683 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:25.383683 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:25.383683 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:25.383683 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:25.383683 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:25.383683 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:25.383683 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:25.384628 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:25.384628 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:25.384628 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 15:01:25.384628 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 15:01:25.384628 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 15:01:25.384628 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 15:01:25.384628 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 15:01:25.384628 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 15:01:25.384628 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 27 15:01:25.384628 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:25.385469 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 337/661] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 15:01:25 2022] GET /identity/v3/users/neutron => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:25.386408 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac0567-7970-4bb9-9325-e6abbd07542b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:25.391778 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e6eaa-30a8-4a22-8f6b-b568b3e461e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:25.392385 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e6eaa-30a8-4a22-8f6b-b568b3e461e0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:25.392666 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e6eaa-30a8-4a22-8f6b-b568b3e461e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:25.392943 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e6eaa-30a8-4a22-8f6b-b568b3e461e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:25.395987 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972e6eaa-30a8-4a22-8f6b-b568b3e461e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:25.395987 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972e6eaa-30a8-4a22-8f6b-b568b3e461e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:25.395987 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972e6eaa-30a8-4a22-8f6b-b568b3e461e0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:25.400494 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:25.409667 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 325/662] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:25.410486 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e6eaa-30a8-4a22-8f6b-b568b3e461e0 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:25.473180 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:25.500398 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac0567-7970-4bb9-9325-e6abbd07542b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:25.517840 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e6eaa-30a8-4a22-8f6b-b568b3e461e0 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:25.535303 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc83944-8583-421b-a4cf-221f851f2898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:25.536099 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc83944-8583-421b-a4cf-221f851f2898 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:25.536379 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc83944-8583-421b-a4cf-221f851f2898 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:25.536653 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc83944-8583-421b-a4cf-221f851f2898 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:25.547551 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc83944-8583-421b-a4cf-221f851f2898 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:25.548999 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc83944-8583-421b-a4cf-221f851f2898 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:25.552934 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d334c9-a5e6-4bbf-8df0-f4c3e34edb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:25.553724 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d334c9-a5e6-4bbf-8df0-f4c3e34edb58 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:25.554011 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d334c9-a5e6-4bbf-8df0-f4c3e34edb58 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:25.554285 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d334c9-a5e6-4bbf-8df0-f4c3e34edb58 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:25.564306 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d334c9-a5e6-4bbf-8df0-f4c3e34edb58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:01:25.566136 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-8cc83944-8583-421b-a4cf-221f851f2898 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:25.566136 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:25.566136 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:25.566136 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:25.566136 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:25.566136 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:25.566136 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:25.566136 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:25.566136 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:25.566136 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:25.566136 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:25.566136 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:25.566136 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:25.566136 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:25.566136 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:25.567201 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:25.567201 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:25.567201 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:25.567201 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:25.567201 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:25.567201 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:25.567201 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:25.567201 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:25.567201 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:25.567201 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:25.567201 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:25.567201 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:25.567201 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:25.567201 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:25.567201 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:25.567201 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:25.567201 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:25.568093 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:25.568093 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:25.568093 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:25.568093 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:25.568093 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 338/663] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:25.568093 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc83944-8583-421b-a4cf-221f851f2898 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:25.572188 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-46d334c9-a5e6-4bbf-8df0-f4c3e34edb58 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 15:01:25.572188 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:25.572188 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:25.572188 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:25.572188 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:25.572188 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:25.572188 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:25.572188 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:25.572188 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:25.572188 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:25.572188 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:25.572188 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:25.572188 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:01:25.572188 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:01:25.572188 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:01:25.572188 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:01:25.573297 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:01:25.573297 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:01:25.573297 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:01:25.573297 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:01:25.573297 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:25.573297 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:25.573297 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:25.573297 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:25.573297 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:25.573297 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:25.573297 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:25.573297 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:25.573297 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:25.573297 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:25.573297 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:25.573297 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:25.574270 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:25.574270 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:25.574270 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:25.574270 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:25.574270 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:01:25.574270 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:01:25.574270 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:01:25.574270 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:01:25.574270 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:01:25.574270 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:01:25.574270 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 15:01:25.574270 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:25.574270 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 326/664] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 15:01:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:25.576229 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d334c9-a5e6-4bbf-8df0-f4c3e34edb58 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:25.630122 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:25.666821 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc83944-8583-421b-a4cf-221f851f2898 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:25.677931 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d334c9-a5e6-4bbf-8df0-f4c3e34edb58 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:25.694705 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:25.700072 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a74f0-cb71-4495-bef2-a4b47648a9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:25.700722 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a74f0-cb71-4495-bef2-a4b47648a9dc None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:25.701005 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a74f0-cb71-4495-bef2-a4b47648a9dc None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:25.701285 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a74f0-cb71-4495-bef2-a4b47648a9dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:25.702606 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a74f0-cb71-4495-bef2-a4b47648a9dc None admin] RBAC: Adding query filter params (role.id=13f37584013544e7ad1e2c8f350da6d2, scope.project.id=daf12585428b41a2b45bdbc85413a6c4, user.id=451a9232f95e494683d81b6078a318d0) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:25.702953 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a74f0-cb71-4495-bef2-a4b47648a9dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:25.704482 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a74f0-cb71-4495-bef2-a4b47648a9dc None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:25.710223 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 339/665] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:01:25 2022] GET /identity/v3/role_assignments?user.id=451a9232f95e494683d81b6078a318d0&scope.project.id=daf12585428b41a2b45bdbc85413a6c4&role.id=13f37584013544e7ad1e2c8f350da6d2 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:25.711024 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a74f0-cb71-4495-bef2-a4b47648a9dc None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:25.712868 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0638e4aa-4d46-4819-bb0c-f564ac277302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:25.713798 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638e4aa-4d46-4819-bb0c-f564ac277302 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:25.714085 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638e4aa-4d46-4819-bb0c-f564ac277302 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:25.714367 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638e4aa-4d46-4819-bb0c-f564ac277302 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:25.715248 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0638e4aa-4d46-4819-bb0c-f564ac277302 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:25.715562 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0638e4aa-4d46-4819-bb0c-f564ac277302 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:25.717045 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0638e4aa-4d46-4819-bb0c-f564ac277302 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:25.738611 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 327/666] 10.222.0.50 () {56 vars in 1139 bytes} [Fri May 27 15:01:25 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:25.739932 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0638e4aa-4d46-4819-bb0c-f564ac277302 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:25.867745 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0638e4aa-4d46-4819-bb0c-f564ac277302 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:25.869392 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a74f0-cb71-4495-bef2-a4b47648a9dc None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:25.901840 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:25.908386 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b17f3b-77b3-4acb-a807-556d92eee294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:25.909071 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b17f3b-77b3-4acb-a807-556d92eee294 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:25.910576 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b17f3b-77b3-4acb-a807-556d92eee294 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:25.910576 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b17f3b-77b3-4acb-a807-556d92eee294 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:25.910576 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b17f3b-77b3-4acb-a807-556d92eee294 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:25.910830 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b17f3b-77b3-4acb-a807-556d92eee294 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:25.912063 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ae83f-2777-4dbd-a39c-4eae9afd6052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:25.912185 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b17f3b-77b3-4acb-a807-556d92eee294 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:25.912755 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ae83f-2777-4dbd-a39c-4eae9afd6052 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:25.913052 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ae83f-2777-4dbd-a39c-4eae9afd6052 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:25.913311 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ae83f-2777-4dbd-a39c-4eae9afd6052 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:25.921018 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79ae83f-2777-4dbd-a39c-4eae9afd6052 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:25.922880 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79ae83f-2777-4dbd-a39c-4eae9afd6052 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:25.924655 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 328/667] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:25.925575 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b17f3b-77b3-4acb-a807-556d92eee294 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:25.937873 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-c79ae83f-2777-4dbd-a39c-4eae9afd6052 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:25.937873 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:25.937873 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:25.937873 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:25.937873 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:25.937873 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:25.937873 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:25.937873 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:25.937873 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:25.937873 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:25.937873 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:25.937873 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:25.937873 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 15:01:25.937873 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 15:01:25.937873 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 15:01:25.938985 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 15:01:25.938985 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:25.938985 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:25.938985 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:25.938985 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:25.938985 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:25.938985 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:25.938985 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:25.938985 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:25.938985 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:25.938985 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:25.938985 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:25.938985 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:25.938985 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:25.938985 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:25.938985 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:25.938985 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:25.940119 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 15:01:25.940119 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 15:01:25.940119 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 15:01:25.940119 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 15:01:25.940119 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 15:01:25.940119 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 15:01:25.940119 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:25.940119 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:25.940119 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 340/668] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:01:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:25.940119 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ae83f-2777-4dbd-a39c-4eae9afd6052 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:25.974460 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:26.046004 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b17f3b-77b3-4acb-a807-556d92eee294 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:26.056414 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ae83f-2777-4dbd-a39c-4eae9afd6052 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:26.081722 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f3e9eb-cb38-4285-9e96-f47da49787a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:26.082500 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f3e9eb-cb38-4285-9e96-f47da49787a9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:26.082784 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f3e9eb-cb38-4285-9e96-f47da49787a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:26.083209 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f3e9eb-cb38-4285-9e96-f47da49787a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:26.083948 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f3e9eb-cb38-4285-9e96-f47da49787a9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:26.084254 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f3e9eb-cb38-4285-9e96-f47da49787a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:26.085729 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f3e9eb-cb38-4285-9e96-f47da49787a9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:26.091242 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c75af-8b60-4fa0-8071-2cc32343e6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:26.091896 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c75af-8b60-4fa0-8071-2cc32343e6fd None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:26.092183 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c75af-8b60-4fa0-8071-2cc32343e6fd None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:26.092463 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c75af-8b60-4fa0-8071-2cc32343e6fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:26.099138 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 329/669] 10.222.0.50 () {56 vars in 1103 bytes} [Fri May 27 15:01:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:26.099138 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f3e9eb-cb38-4285-9e96-f47da49787a9 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:26.159727 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:26.179068 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85c75af-8b60-4fa0-8071-2cc32343e6fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:26.180453 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85c75af-8b60-4fa0-8071-2cc32343e6fd None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:26.218768 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:26.222806 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f3e9eb-cb38-4285-9e96-f47da49787a9 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:26.259032 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d43c88-3d54-4be8-aaff-726333db2bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:26.259946 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d43c88-3d54-4be8-aaff-726333db2bfb None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:26.260230 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d43c88-3d54-4be8-aaff-726333db2bfb None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:26.260505 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d43c88-3d54-4be8-aaff-726333db2bfb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:26.269613 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d43c88-3d54-4be8-aaff-726333db2bfb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:26.271397 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d43c88-3d54-4be8-aaff-726333db2bfb None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:26.290051 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 341/670] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:01:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 15:01:26.290051 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c75af-8b60-4fa0-8071-2cc32343e6fd None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:26.290318 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-80d43c88-3d54-4be8-aaff-726333db2bfb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:26.290318 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:26.290318 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:26.290318 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:26.290318 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:26.290318 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:26.290318 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:26.290318 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:26.290318 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:26.290318 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:26.290318 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:26.290318 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:26.290318 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 15:01:26.290318 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 15:01:26.290318 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 15:01:26.291604 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 15:01:26.291604 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:26.291604 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:26.291604 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:26.291604 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:26.291604 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:26.291604 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:26.291604 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:26.291604 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:26.291604 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:26.291604 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:26.291604 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:26.291604 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:26.291604 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:26.291604 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:26.291604 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:26.291604 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:26.292520 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 15:01:26.292520 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 15:01:26.292520 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 15:01:26.292520 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 15:01:26.292520 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 15:01:26.292520 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 15:01:26.292520 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:26.292520 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:26.292520 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 330/671] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:01:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:26.292520 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d43c88-3d54-4be8-aaff-726333db2bfb None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:26.383597 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:26.397091 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d43c88-3d54-4be8-aaff-726333db2bfb None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:26.397767 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c75af-8b60-4fa0-8071-2cc32343e6fd None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:26.432535 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ce869-4162-467c-b20c-101b1ac7b6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:26.433263 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ce869-4162-467c-b20c-101b1ac7b6b5 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:26.433548 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ce869-4162-467c-b20c-101b1ac7b6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:26.433822 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ce869-4162-467c-b20c-101b1ac7b6b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:26.434124 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c2cebc-ac42-4ca5-97d0-533e45bb33ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:26.434777 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c2cebc-ac42-4ca5-97d0-533e45bb33ea None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:26.435087 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c2cebc-ac42-4ca5-97d0-533e45bb33ea None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:26.435363 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c2cebc-ac42-4ca5-97d0-533e45bb33ea None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:26.436135 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c2cebc-ac42-4ca5-97d0-533e45bb33ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:26.436466 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c2cebc-ac42-4ca5-97d0-533e45bb33ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:26.438495 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c2cebc-ac42-4ca5-97d0-533e45bb33ea None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:26.444900 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ce869-4162-467c-b20c-101b1ac7b6b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:26.446352 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ce869-4162-467c-b20c-101b1ac7b6b5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:26.455940 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 342/672] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 15:01:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:26.455940 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c2cebc-ac42-4ca5-97d0-533e45bb33ea None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:26.470474 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-3b0ce869-4162-467c-b20c-101b1ac7b6b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:26.470474 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:26.470474 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:26.470474 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:26.470474 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:26.470474 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:26.470474 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:26.470474 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:26.470474 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:26.470474 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:26.470474 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:26.470474 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:26.470474 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:26.470474 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:26.470474 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:26.471894 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:26.471894 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:26.471894 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:26.471894 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:26.471894 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:26.471894 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:26.471894 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:26.471894 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:26.471894 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:26.471894 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:26.471894 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:26.471894 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:26.471894 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:26.471894 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:26.471894 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:26.471894 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:26.471894 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:26.473668 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:26.473668 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:26.473668 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:26.473668 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:26.475152 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 331/673] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:26.476688 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ce869-4162-467c-b20c-101b1ac7b6b5 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:26.492866 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:26.571928 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c2cebc-ac42-4ca5-97d0-533e45bb33ea None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:26.592194 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ce869-4162-467c-b20c-101b1ac7b6b5 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:26.611464 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f933a0-fd2e-4e0a-b060-402914b08a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:26.612572 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f933a0-fd2e-4e0a-b060-402914b08a27 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:26.613165 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f933a0-fd2e-4e0a-b060-402914b08a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:26.613446 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f933a0-fd2e-4e0a-b060-402914b08a27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:26.626019 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f933a0-fd2e-4e0a-b060-402914b08a27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:26.627476 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd44340-8f4a-489a-ab7c-e0d4d592e2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:26.627626 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f933a0-fd2e-4e0a-b060-402914b08a27 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:26.628141 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd44340-8f4a-489a-ab7c-e0d4d592e2ef None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:26.628428 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd44340-8f4a-489a-ab7c-e0d4d592e2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:26.628702 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd44340-8f4a-489a-ab7c-e0d4d592e2ef None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:26.629481 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd44340-8f4a-489a-ab7c-e0d4d592e2ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:26.629789 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd44340-8f4a-489a-ab7c-e0d4d592e2ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:26.631222 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd44340-8f4a-489a-ab7c-e0d4d592e2ef None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:26.635990 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:26.647653 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 332/674] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 15:01:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:26.647653 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd44340-8f4a-489a-ab7c-e0d4d592e2ef None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:26.647884 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-c4f933a0-fd2e-4e0a-b060-402914b08a27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:26.647884 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:26.647884 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:26.647884 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:26.647884 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:26.647884 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:26.647884 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:26.647884 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:26.647884 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:26.647884 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:26.647884 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:26.647884 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:26.647884 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:26.647884 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:26.647884 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:26.648754 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:26.648754 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:26.648754 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:26.648754 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:26.648754 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:26.648754 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:26.648754 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:26.648754 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:26.648754 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:26.648754 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:26.648754 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:26.648754 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:26.648754 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:26.648754 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:26.648754 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:26.648754 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:26.648754 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:26.649678 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:26.649678 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:26.649678 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:26.649678 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:26.649678 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 343/675] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:26.649678 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f933a0-fd2e-4e0a-b060-402914b08a27 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:26.719399 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:26.752165 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd44340-8f4a-489a-ab7c-e0d4d592e2ef None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:26.758410 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f933a0-fd2e-4e0a-b060-402914b08a27 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:26.785727 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc27101-750e-4b95-9231-974931445ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:26.786373 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc27101-750e-4b95-9231-974931445ef0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:26.786658 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc27101-750e-4b95-9231-974931445ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:26.786977 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc27101-750e-4b95-9231-974931445ef0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:26.787776 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc27101-750e-4b95-9231-974931445ef0 None admin] RBAC: Adding query filter params (role.id=13f37584013544e7ad1e2c8f350da6d2, scope.project.id=daf12585428b41a2b45bdbc85413a6c4, user.id=1afda9d6b1d64e6ab400b26124cf1290) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:26.788107 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc27101-750e-4b95-9231-974931445ef0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:26.789500 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc27101-750e-4b95-9231-974931445ef0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:26.792539 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d96309-4aa5-48a4-b11c-a52dfc3a53cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:26.793194 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d96309-4aa5-48a4-b11c-a52dfc3a53cb None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:26.793479 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d96309-4aa5-48a4-b11c-a52dfc3a53cb None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:26.793808 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d96309-4aa5-48a4-b11c-a52dfc3a53cb None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:26.794458 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d96309-4aa5-48a4-b11c-a52dfc3a53cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:26.794765 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d96309-4aa5-48a4-b11c-a52dfc3a53cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:26.796757 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d96309-4aa5-48a4-b11c-a52dfc3a53cb None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:26.804767 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 333/676] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:01:26 2022] GET /identity/v3/role_assignments?user.id=1afda9d6b1d64e6ab400b26124cf1290&scope.project.id=daf12585428b41a2b45bdbc85413a6c4&role.id=13f37584013544e7ad1e2c8f350da6d2 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:26.804767 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc27101-750e-4b95-9231-974931445ef0 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:26.813091 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 344/677] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:26.814321 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d96309-4aa5-48a4-b11c-a52dfc3a53cb None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:26.898449 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:26.950765 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc27101-750e-4b95-9231-974931445ef0 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:26.956348 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d96309-4aa5-48a4-b11c-a52dfc3a53cb None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:26.989998 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df164b02-f235-4d5c-a6d3-76e093b5d9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:26.990916 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df164b02-f235-4d5c-a6d3-76e093b5d9ad None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:26.991360 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df164b02-f235-4d5c-a6d3-76e093b5d9ad None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:26.991773 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df164b02-f235-4d5c-a6d3-76e093b5d9ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:26.992731 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df164b02-f235-4d5c-a6d3-76e093b5d9ad None admin] RBAC: Adding query filter params (role.id=13f37584013544e7ad1e2c8f350da6d2, scope.project.id=daf12585428b41a2b45bdbc85413a6c4, user.id=896af6ccf89847e5903f6a4fe3d9d810) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:26.993194 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df164b02-f235-4d5c-a6d3-76e093b5d9ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:26.994951 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df164b02-f235-4d5c-a6d3-76e093b5d9ad None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:26.998819 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:27.001309 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e6ad41-cbf4-45a3-9e81-64a0ff13f91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:27.002023 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e6ad41-cbf4-45a3-9e81-64a0ff13f91b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:27.003001 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e6ad41-cbf4-45a3-9e81-64a0ff13f91b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:27.003001 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e6ad41-cbf4-45a3-9e81-64a0ff13f91b None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:27.003305 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e6ad41-cbf4-45a3-9e81-64a0ff13f91b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:27.003612 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e6ad41-cbf4-45a3-9e81-64a0ff13f91b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:27.005013 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e6ad41-cbf4-45a3-9e81-64a0ff13f91b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:27.006371 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 334/678] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:01:26 2022] GET /identity/v3/role_assignments?user.id=896af6ccf89847e5903f6a4fe3d9d810&scope.project.id=daf12585428b41a2b45bdbc85413a6c4&role.id=13f37584013544e7ad1e2c8f350da6d2 => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:27.007756 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df164b02-f235-4d5c-a6d3-76e093b5d9ad None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:27.020600 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 345/679] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:27.026646 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e6ad41-cbf4-45a3-9e81-64a0ff13f91b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:27.135771 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df164b02-f235-4d5c-a6d3-76e093b5d9ad None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:27.145902 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e6ad41-cbf4-45a3-9e81-64a0ff13f91b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:27.168136 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:27.180059 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e1bf9b-1f9c-4725-887a-cf9b742e847d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:27.181360 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e1bf9b-1f9c-4725-887a-cf9b742e847d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:27.181649 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e1bf9b-1f9c-4725-887a-cf9b742e847d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:27.181934 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e1bf9b-1f9c-4725-887a-cf9b742e847d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:27.188443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0f03eb-009f-41f0-944d-a0fba94e4513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:27.189634 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0f03eb-009f-41f0-944d-a0fba94e4513 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:27.190143 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0f03eb-009f-41f0-944d-a0fba94e4513 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:27.190620 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0f03eb-009f-41f0-944d-a0fba94e4513 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:27.191999 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e1bf9b-1f9c-4725-887a-cf9b742e847d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:27.193613 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e1bf9b-1f9c-4725-887a-cf9b742e847d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:27.199931 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0f03eb-009f-41f0-944d-a0fba94e4513 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:27.201712 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0f03eb-009f-41f0-944d-a0fba94e4513 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:27.210395 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-17e1bf9b-1f9c-4725-887a-cf9b742e847d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 15:01:27.210395 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:27.210395 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:27.210395 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:27.210395 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:27.210395 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:27.210395 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:27.210395 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:27.210395 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:27.210395 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:27.210395 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:27.210395 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:27.210395 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 15:01:27.210395 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 15:01:27.210395 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 15:01:27.210395 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 15:01:27.211457 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:27.211457 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:27.211457 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 15:01:27.211457 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:27.211457 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 15:01:27.211457 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:27.211457 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:27.211457 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:27.211457 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:27.211457 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:27.211457 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:27.211457 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:27.211457 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:27.211457 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:27.211457 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:27.211457 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:27.211457 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:27.211457 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:27.212398 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:27.212398 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:27.212398 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 15:01:27.212398 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 15:01:27.212398 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 15:01:27.212398 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 15:01:27.212398 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 15:01:27.212398 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 15:01:27.212398 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 15:01:27.212398 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:27.212398 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 335/680] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 15:01:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:27.218528 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e1bf9b-1f9c-4725-887a-cf9b742e847d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:27.222431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-2d0f03eb-009f-41f0-944d-a0fba94e4513 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:27.222431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:27.222431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:27.222431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:27.222431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:27.222431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:27.222431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:27.222431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:27.222431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:27.222431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:27.222431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:27.222431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:27.222431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 15:01:27.222431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 15:01:27.222431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 15:01:27.223537 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 15:01:27.223537 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:27.223537 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:27.223537 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:27.223537 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:27.223537 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:27.223537 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:27.223537 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:27.223537 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:27.223537 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:27.223537 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:27.223537 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:27.223537 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:27.223537 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:27.223537 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:27.223537 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:27.223537 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:27.224690 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 15:01:27.224690 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 15:01:27.224690 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 15:01:27.224690 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 15:01:27.224690 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 15:01:27.224690 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 15:01:27.224690 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:27.224690 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:27.226426 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 346/681] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:01:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:27.232011 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0f03eb-009f-41f0-944d-a0fba94e4513 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:27.292212 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:27.355018 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e1bf9b-1f9c-4725-887a-cf9b742e847d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:27.359094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0f03eb-009f-41f0-944d-a0fba94e4513 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:27.394822 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65112c13-2ff0-4d0a-86d7-a3272ae9a2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:27.395844 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65112c13-2ff0-4d0a-86d7-a3272ae9a2f0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:27.396270 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65112c13-2ff0-4d0a-86d7-a3272ae9a2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:27.396824 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65112c13-2ff0-4d0a-86d7-a3272ae9a2f0 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:27.397761 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65112c13-2ff0-4d0a-86d7-a3272ae9a2f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:27.398210 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65112c13-2ff0-4d0a-86d7-a3272ae9a2f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:27.398610 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e07f796-c979-4890-9ab0-b53cf70f1cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:27.399434 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07f796-c979-4890-9ab0-b53cf70f1cbb None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:27.399818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07f796-c979-4890-9ab0-b53cf70f1cbb None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:27.400197 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07f796-c979-4890-9ab0-b53cf70f1cbb None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:27.400462 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65112c13-2ff0-4d0a-86d7-a3272ae9a2f0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:27.401230 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e07f796-c979-4890-9ab0-b53cf70f1cbb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:27.401643 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e07f796-c979-4890-9ab0-b53cf70f1cbb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:27.403415 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e07f796-c979-4890-9ab0-b53cf70f1cbb None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:27.416638 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 347/682] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 15:01:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:27.431269 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 336/683] 10.222.0.50 () {56 vars in 1135 bytes} [Fri May 27 15:01:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:01:27.435911 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65112c13-2ff0-4d0a-86d7-a3272ae9a2f0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:27.439016 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e07f796-c979-4890-9ab0-b53cf70f1cbb None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:27.499300 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:27.630769 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65112c13-2ff0-4d0a-86d7-a3272ae9a2f0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:27.686140 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee78bed9-555a-47db-b371-560886ce5907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:27.687364 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee78bed9-555a-47db-b371-560886ce5907 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:27.687638 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee78bed9-555a-47db-b371-560886ce5907 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:27.687907 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee78bed9-555a-47db-b371-560886ce5907 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:27.688737 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee78bed9-555a-47db-b371-560886ce5907 None admin] RBAC: Adding query filter params (role.id=13f37584013544e7ad1e2c8f350da6d2, scope.project.id=daf12585428b41a2b45bdbc85413a6c4, user.id=f874edc08438434189f1d14d79291487) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:27.689034 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee78bed9-555a-47db-b371-560886ce5907 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:27.690459 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee78bed9-555a-47db-b371-560886ce5907 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:27.697015 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 348/684] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:01:27 2022] GET /identity/v3/role_assignments?user.id=f874edc08438434189f1d14d79291487&scope.project.id=daf12585428b41a2b45bdbc85413a6c4&role.id=13f37584013544e7ad1e2c8f350da6d2 => generated 608 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:27.746419 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e07f796-c979-4890-9ab0-b53cf70f1cbb None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:27.765703 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:27.790814 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc243467-6918-4ce1-bbfd-c9fd770f0a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:27.791511 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc243467-6918-4ce1-bbfd-c9fd770f0a24 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:27.791791 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc243467-6918-4ce1-bbfd-c9fd770f0a24 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:27.792064 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc243467-6918-4ce1-bbfd-c9fd770f0a24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:27.870607 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc243467-6918-4ce1-bbfd-c9fd770f0a24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:27.872116 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc243467-6918-4ce1-bbfd-c9fd770f0a24 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:27.982671 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 337/685] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:01:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 15:01:27.988920 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee78bed9-555a-47db-b371-560886ce5907 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:28.097962 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee78bed9-555a-47db-b371-560886ce5907 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:28.100335 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:28.135276 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42da67b-6555-4c2b-a3f3-f497d0a8c1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:28.136082 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42da67b-6555-4c2b-a3f3-f497d0a8c1c3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:28.136366 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42da67b-6555-4c2b-a3f3-f497d0a8c1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:28.136645 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42da67b-6555-4c2b-a3f3-f497d0a8c1c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:28.151789 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42da67b-6555-4c2b-a3f3-f497d0a8c1c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:28.153998 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42da67b-6555-4c2b-a3f3-f497d0a8c1c3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:28.174476 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-a42da67b-6555-4c2b-a3f3-f497d0a8c1c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:28.174476 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:28.174476 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:28.174476 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:28.174476 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:28.174476 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:28.174476 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:28.174476 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:28.174476 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:28.174476 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:28.174476 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:28.174476 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:28.174476 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:28.174476 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:28.174476 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:28.175434 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:28.175434 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:28.175434 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:28.175434 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:28.175434 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:28.175434 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:28.175434 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:28.175434 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:28.175434 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:28.175434 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:28.175434 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:28.175434 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:28.175434 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:28.175434 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:28.175434 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:28.175434 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:28.175434 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:28.176338 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:28.176338 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:28.176338 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:28.176338 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:28.178550 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 349/686] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:28.183782 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc243467-6918-4ce1-bbfd-c9fd770f0a24 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:28.192150 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ee230-0567-4902-8899-9aab1b80014c None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:28.192447 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ee230-0567-4902-8899-9aab1b80014c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:28.192715 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ee230-0567-4902-8899-9aab1b80014c None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:28.194370 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 350/687] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 15:01:28.199101 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0aad7-72b1-4d5e-9dee-a354bbbba1aa None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:28.199554 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0aad7-72b1-4d5e-9dee-a354bbbba1aa None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:28.200008 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0aad7-72b1-4d5e-9dee-a354bbbba1aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:28.211396 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:28.222483 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-23c0aad7-72b1-4d5e-9dee-a354bbbba1aa None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:28.223828 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-23c0aad7-72b1-4d5e-9dee-a354bbbba1aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:28.297832 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 351/688] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 15:01:28.322278 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0aad7-72b1-4d5e-9dee-a354bbbba1aa None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:28.400908 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc243467-6918-4ce1-bbfd-c9fd770f0a24 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:28.427296 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:28.435419 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0aad7-72b1-4d5e-9dee-a354bbbba1aa None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:28.527205 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7940f8d9-22d0-4c86-b420-f7de5ddf67be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:28.527901 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940f8d9-22d0-4c86-b420-f7de5ddf67be None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:28.528182 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940f8d9-22d0-4c86-b420-f7de5ddf67be None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:28.528465 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940f8d9-22d0-4c86-b420-f7de5ddf67be None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:28.529194 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7940f8d9-22d0-4c86-b420-f7de5ddf67be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:28.529497 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7940f8d9-22d0-4c86-b420-f7de5ddf67be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:28.531349 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7940f8d9-22d0-4c86-b420-f7de5ddf67be None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:28.551182 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 338/689] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:28.560860 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7940f8d9-22d0-4c86-b420-f7de5ddf67be None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:28.605265 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a2ebdf-0fcc-4546-9a88-faa65eef274f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:28.606160 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2ebdf-0fcc-4546-9a88-faa65eef274f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:28.606451 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2ebdf-0fcc-4546-9a88-faa65eef274f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:28.606728 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a2ebdf-0fcc-4546-9a88-faa65eef274f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:28.620569 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a2ebdf-0fcc-4546-9a88-faa65eef274f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:01:28.630263 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-86a2ebdf-0fcc-4546-9a88-faa65eef274f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:01:28.630263 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:28.630263 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:28.630263 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:28.630263 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:28.630263 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:28.630263 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:28.630263 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:28.630263 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:28.630263 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:28.630263 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:28.630263 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:28.630263 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:01:28.630263 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:01:28.630263 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:01:28.630263 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:01:28.631349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:01:28.631349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:01:28.631349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:01:28.631349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:01:28.631349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:28.631349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:28.631349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:28.631349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:28.631349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:28.631349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:28.631349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:28.631349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:28.631349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:28.631349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:28.631349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:28.631349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:28.632221 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:28.632221 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:28.632221 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:28.632221 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:28.632221 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:01:28.632221 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:01:28.632221 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:01:28.632221 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:01:28.632221 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:01:28.632221 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:01:28.632221 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:01:28.632221 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:28.632879 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 352/690] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 15:01:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:01:28.640499 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a2ebdf-0fcc-4546-9a88-faa65eef274f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:28.732705 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:28.782147 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a2ebdf-0fcc-4546-9a88-faa65eef274f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:28.818991 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7940f8d9-22d0-4c86-b420-f7de5ddf67be None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:28.820605 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b304a9-4488-4aeb-803f-58094038ad91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:28.821375 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b304a9-4488-4aeb-803f-58094038ad91 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:28.821732 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b304a9-4488-4aeb-803f-58094038ad91 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:28.822085 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b304a9-4488-4aeb-803f-58094038ad91 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:28.822939 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b304a9-4488-4aeb-803f-58094038ad91 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:28.823320 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b304a9-4488-4aeb-803f-58094038ad91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:28.824764 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b304a9-4488-4aeb-803f-58094038ad91 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:28.837978 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 353/691] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:01:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:28.844545 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b304a9-4488-4aeb-803f-58094038ad91 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:28.864410 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:28.896747 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19911e9-11d1-42e7-8997-b6e5a3b95ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:28.897647 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19911e9-11d1-42e7-8997-b6e5a3b95ce7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:28.897935 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19911e9-11d1-42e7-8997-b6e5a3b95ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:28.898212 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19911e9-11d1-42e7-8997-b6e5a3b95ce7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:28.914278 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19911e9-11d1-42e7-8997-b6e5a3b95ce7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:28.915767 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19911e9-11d1-42e7-8997-b6e5a3b95ce7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:28.937176 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-a19911e9-11d1-42e7-8997-b6e5a3b95ce7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:28.937176 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:28.937176 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:28.937176 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:28.937176 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:28.937176 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:28.937176 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:28.937176 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:28.937176 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:28.937176 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:28.937176 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:28.937176 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:28.937176 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 15:01:28.937176 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 15:01:28.937176 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 15:01:28.938191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 15:01:28.938191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:28.938191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:28.938191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:28.938191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:28.938191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:28.938191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:28.938191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:28.938191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:28.938191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:28.938191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:28.938191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:28.938191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:28.938191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:28.938191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:28.938191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:28.938191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:28.939150 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 15:01:28.939150 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 15:01:28.939150 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 15:01:28.939150 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 15:01:28.939150 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 15:01:28.939150 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 15:01:28.939150 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:28.939150 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:28.939150 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 339/692] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:01:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:28.944971 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19911e9-11d1-42e7-8997-b6e5a3b95ce7 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:29.056911 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b304a9-4488-4aeb-803f-58094038ad91 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:29.059443 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19911e9-11d1-42e7-8997-b6e5a3b95ce7 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:29.098681 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d67404-91ff-47c5-9bfc-cc06814d3352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:29.099293 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d67404-91ff-47c5-9bfc-cc06814d3352 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:29.099579 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d67404-91ff-47c5-9bfc-cc06814d3352 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:29.099860 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d67404-91ff-47c5-9bfc-cc06814d3352 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:29.100650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d67404-91ff-47c5-9bfc-cc06814d3352 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:29.100961 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d67404-91ff-47c5-9bfc-cc06814d3352 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:29.102399 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d67404-91ff-47c5-9bfc-cc06814d3352 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:29.105256 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:29.122788 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 340/693] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 15:01:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:01:29.125064 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d67404-91ff-47c5-9bfc-cc06814d3352 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:29.148485 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f3597-750c-432f-9ddd-ce8f14032425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:29.149409 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f3597-750c-432f-9ddd-ce8f14032425 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:29.149696 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f3597-750c-432f-9ddd-ce8f14032425 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:29.149974 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f3597-750c-432f-9ddd-ce8f14032425 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:29.176733 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619f3597-750c-432f-9ddd-ce8f14032425 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:29.178213 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619f3597-750c-432f-9ddd-ce8f14032425 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:29.215954 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-619f3597-750c-432f-9ddd-ce8f14032425 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:29.215954 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:29.215954 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:29.215954 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:29.215954 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:29.215954 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:29.215954 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:29.215954 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:29.215954 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:29.215954 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:29.215954 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:29.215954 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:29.215954 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:29.215954 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:29.215954 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:29.218349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:29.218349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:29.218349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:29.218349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:29.218349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:29.218349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:29.218349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:29.218349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:29.218349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:29.218349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:29.218349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:29.218349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:29.218349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:29.218349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:29.218349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:29.218349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:29.218349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:29.220595 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:29.220595 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:29.220595 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:29.220595 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:29.220595 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 354/694] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:29.226688 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f3597-750c-432f-9ddd-ce8f14032425 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:29.235467 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d67404-91ff-47c5-9bfc-cc06814d3352 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:29.276294 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aadf932-093e-4808-838b-c6a356c2134f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:29.276941 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aadf932-093e-4808-838b-c6a356c2134f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:29.277226 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aadf932-093e-4808-838b-c6a356c2134f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:29.277504 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aadf932-093e-4808-838b-c6a356c2134f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:29.278330 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aadf932-093e-4808-838b-c6a356c2134f None admin] RBAC: Adding query filter params (role.id=13f37584013544e7ad1e2c8f350da6d2, scope.project.id=daf12585428b41a2b45bdbc85413a6c4, user.id=18903f2eab7c415096e075d47f0b9a4d) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:29.278644 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aadf932-093e-4808-838b-c6a356c2134f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:29.280458 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aadf932-093e-4808-838b-c6a356c2134f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:29.295201 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 341/695] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:01:29 2022] GET /identity/v3/role_assignments?user.id=18903f2eab7c415096e075d47f0b9a4d&scope.project.id=daf12585428b41a2b45bdbc85413a6c4&role.id=13f37584013544e7ad1e2c8f350da6d2 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:01:29.367076 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:29.428609 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f3597-750c-432f-9ddd-ce8f14032425 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:29.481088 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe58337-5ae8-4090-9c85-77fda8e6dd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:29.481088 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe58337-5ae8-4090-9c85-77fda8e6dd2a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:29.481088 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe58337-5ae8-4090-9c85-77fda8e6dd2a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:29.481088 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe58337-5ae8-4090-9c85-77fda8e6dd2a None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:29.481894 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe58337-5ae8-4090-9c85-77fda8e6dd2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:29.481894 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe58337-5ae8-4090-9c85-77fda8e6dd2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:29.481894 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe58337-5ae8-4090-9c85-77fda8e6dd2a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:29.497525 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 355/696] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:01:29.503875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aadf932-093e-4808-838b-c6a356c2134f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:29.581981 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:29.625154 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aadf932-093e-4808-838b-c6a356c2134f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:29.657124 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26febd8-bf87-478f-8701-866eb87d8137 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:29.657745 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26febd8-bf87-478f-8701-866eb87d8137 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:29.658226 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26febd8-bf87-478f-8701-866eb87d8137 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:29.660136 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 356/697] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:29.665960 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c182d7a8-2cd8-45af-a909-7c4d16a5498d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:29.666569 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c182d7a8-2cd8-45af-a909-7c4d16a5498d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:29.666677 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea653e2-0f34-4473-a18a-dac803f0fd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:29.667242 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c182d7a8-2cd8-45af-a909-7c4d16a5498d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:29.668597 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea653e2-0f34-4473-a18a-dac803f0fd2c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:29.668597 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea653e2-0f34-4473-a18a-dac803f0fd2c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:29.668597 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea653e2-0f34-4473-a18a-dac803f0fd2c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:29.678008 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea653e2-0f34-4473-a18a-dac803f0fd2c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:29.679540 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea653e2-0f34-4473-a18a-dac803f0fd2c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:29.691026 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c182d7a8-2cd8-45af-a909-7c4d16a5498d None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:29.692665 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c182d7a8-2cd8-45af-a909-7c4d16a5498d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:29.697040 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-7ea653e2-0f34-4473-a18a-dac803f0fd2c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 15:01:29.697040 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:29.697040 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:29.697040 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:29.697040 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:29.697040 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:29.697040 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:29.697040 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:29.697040 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:29.697040 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:29.697040 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:29.697040 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:29.697040 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 15:01:29.697040 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 15:01:29.697040 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 15:01:29.697040 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 15:01:29.698045 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:29.698045 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:29.698045 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 15:01:29.698045 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:29.698045 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 15:01:29.698045 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:29.698045 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:29.698045 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:29.698045 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:29.698045 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:29.698045 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:29.698045 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:29.698045 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:29.698045 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:29.698045 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:29.698045 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:29.698045 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:29.698045 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:29.699034 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:29.699034 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:29.699034 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 15:01:29.699034 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 15:01:29.699034 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 15:01:29.699034 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 15:01:29.699034 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 15:01:29.699034 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 15:01:29.699034 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 15:01:29.699034 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:29.699034 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 342/698] 10.222.0.50 () {56 vars in 1093 bytes} [Fri May 27 15:01:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:29.703755 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea653e2-0f34-4473-a18a-dac803f0fd2c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:29.790227 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 357/699] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 15:01:29.819417 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c182d7a8-2cd8-45af-a909-7c4d16a5498d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:29.841539 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea653e2-0f34-4473-a18a-dac803f0fd2c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:29.868375 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:29.887125 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:29.887795 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d71afd0-7d0a-4a20-b77e-1657931fb4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:29.888648 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d71afd0-7d0a-4a20-b77e-1657931fb4a2 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:29.889083 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d71afd0-7d0a-4a20-b77e-1657931fb4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:29.889722 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d71afd0-7d0a-4a20-b77e-1657931fb4a2 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:29.891187 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d71afd0-7d0a-4a20-b77e-1657931fb4a2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:29.891772 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d71afd0-7d0a-4a20-b77e-1657931fb4a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:29.893239 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d71afd0-7d0a-4a20-b77e-1657931fb4a2 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:29.916008 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 343/700] 10.222.0.50 () {56 vars in 1133 bytes} [Fri May 27 15:01:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:29.918618 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394dacf4-3f15-413b-a8a4-155794875372 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:29.918936 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394dacf4-3f15-413b-a8a4-155794875372 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:29.919209 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394dacf4-3f15-413b-a8a4-155794875372 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:29.921590 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 344/701] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:29.926342 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394dacf4-3f15-413b-a8a4-155794875372 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:29.961820 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c182d7a8-2cd8-45af-a909-7c4d16a5498d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:30.027756 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40be39b1-20c8-49b1-a325-8360797dc4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:30.030001 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be39b1-20c8-49b1-a325-8360797dc4dc None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:30.032380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be39b1-20c8-49b1-a325-8360797dc4dc None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:30.033159 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40be39b1-20c8-49b1-a325-8360797dc4dc None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:30.034691 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40be39b1-20c8-49b1-a325-8360797dc4dc None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:30.038059 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40be39b1-20c8-49b1-a325-8360797dc4dc None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:30.067981 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-40be39b1-20c8-49b1-a325-8360797dc4dc None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 15:01:30.067981 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:30.067981 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 15:01:30.067981 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 15:01:30.067981 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 15:01:30.067981 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 15:01:30.067981 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 15:01:30.067981 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:01:30.067981 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 15:01:30.067981 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:30.067981 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 15:01:30.067981 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:30.067981 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:30.067981 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:30.067981 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:30.067981 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:30.070365 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:30.070365 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:30.070365 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:30.070365 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:30.070365 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:30.070365 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:30.070365 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:30.070365 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 15:01:30.070365 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 15:01:30.070365 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 15:01:30.070365 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 15:01:30.070365 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:30.070365 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:30.070365 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:30.070365 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:30.070365 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:30.072566 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:30.072566 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:30.072566 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:30.072566 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:30.072566 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:30.072566 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:30.072566 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:30.072566 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:30.072566 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:30.072566 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:30.072566 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:30.072566 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 15:01:30.072566 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:01:30.072566 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 15:01:30.072566 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:30.083195 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 358/702] 10.222.0.50 () {56 vars in 1129 bytes} [Fri May 27 15:01:29 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:01:30.084677 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152ae0d-8e82-45df-96b9-bd457ddbb772 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:30.084966 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152ae0d-8e82-45df-96b9-bd457ddbb772 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:30.085245 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152ae0d-8e82-45df-96b9-bd457ddbb772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:30.095875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394dacf4-3f15-413b-a8a4-155794875372 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:30.107691 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5152ae0d-8e82-45df-96b9-bd457ddbb772 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:30.109495 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5152ae0d-8e82-45df-96b9-bd457ddbb772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:30.142666 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f06b0-5b5a-4208-a030-e34bf34426a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:30.143798 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f06b0-5b5a-4208-a030-e34bf34426a6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:30.144347 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f06b0-5b5a-4208-a030-e34bf34426a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:30.144796 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f06b0-5b5a-4208-a030-e34bf34426a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:30.200944 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 359/703] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:01:30.202705 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5152ae0d-8e82-45df-96b9-bd457ddbb772 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:30.220503 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9f06b0-5b5a-4208-a030-e34bf34426a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:30.222123 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9f06b0-5b5a-4208-a030-e34bf34426a6 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:30.239306 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:30.264234 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:30.324025 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5152ae0d-8e82-45df-96b9-bd457ddbb772 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:30.346723 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 345/704] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:01:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 15:01:30.347750 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f06b0-5b5a-4208-a030-e34bf34426a6 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:30.369497 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300638b9-0d1e-4fab-9624-edc160922884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:30.370933 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300638b9-0d1e-4fab-9624-edc160922884 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:30.371400 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300638b9-0d1e-4fab-9624-edc160922884 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:30.371850 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300638b9-0d1e-4fab-9624-edc160922884 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:30.372992 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300638b9-0d1e-4fab-9624-edc160922884 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:30.373712 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300638b9-0d1e-4fab-9624-edc160922884 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:30.376780 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300638b9-0d1e-4fab-9624-edc160922884 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:30.383543 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 360/705] 10.222.0.50 () {56 vars in 1124 bytes} [Fri May 27 15:01:30 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:30.388306 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300638b9-0d1e-4fab-9624-edc160922884 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:30.439560 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:30.469307 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f06b0-5b5a-4208-a030-e34bf34426a6 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:30.507422 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6494db7f-0624-4014-841b-2835d2f901ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:30.508279 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6494db7f-0624-4014-841b-2835d2f901ba None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:30.508565 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6494db7f-0624-4014-841b-2835d2f901ba None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:30.508839 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6494db7f-0624-4014-841b-2835d2f901ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:30.513936 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300638b9-0d1e-4fab-9624-edc160922884 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:30.520136 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6494db7f-0624-4014-841b-2835d2f901ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:30.521587 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6494db7f-0624-4014-841b-2835d2f901ba None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:30.540032 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-6494db7f-0624-4014-841b-2835d2f901ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:30.540032 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:30.540032 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:30.540032 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:30.540032 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:30.540032 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:30.540032 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:30.540032 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:30.540032 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:30.540032 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:30.540032 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:30.540032 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:30.540032 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:30.540032 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:30.540032 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:30.540987 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:30.540987 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:30.540987 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:30.540987 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:30.540987 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:30.540987 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:30.540987 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:30.540987 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:30.540987 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:30.540987 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:30.540987 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:30.540987 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:30.540987 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:30.540987 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:30.540987 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:30.540987 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:30.540987 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:30.541872 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:30.541872 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:30.541872 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:30.541872 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:30.541872 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 346/706] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:30.542577 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6494db7f-0624-4014-841b-2835d2f901ba None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:30.552011 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:30.567259 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e9d4b-ce6c-4747-8d5a-41b0bc392305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:30.568455 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e9d4b-ce6c-4747-8d5a-41b0bc392305 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:30.568948 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e9d4b-ce6c-4747-8d5a-41b0bc392305 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:30.569426 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e9d4b-ce6c-4747-8d5a-41b0bc392305 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:30.580916 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2e9d4b-ce6c-4747-8d5a-41b0bc392305 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:30.582649 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2e9d4b-ce6c-4747-8d5a-41b0bc392305 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:30.602015 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-8a2e9d4b-ce6c-4747-8d5a-41b0bc392305 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:30.602015 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:30.602015 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:30.602015 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:30.602015 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:30.602015 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:30.602015 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:30.602015 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:30.602015 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:30.602015 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:30.602015 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:30.602015 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:30.602015 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:30.602015 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:30.602015 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:30.602982 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:30.602982 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:30.602982 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:30.602982 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:30.602982 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:30.602982 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:30.602982 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:30.602982 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:30.602982 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:30.602982 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:30.602982 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:30.602982 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:30.602982 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:30.602982 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:30.602982 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:30.602982 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:30.602982 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:30.603889 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:30.603889 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:30.603889 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:30.603889 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:30.611200 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 361/707] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:30.612351 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e9d4b-ce6c-4747-8d5a-41b0bc392305 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:30.665165 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6494db7f-0624-4014-841b-2835d2f901ba None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:30.696771 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:30.703002 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2db7c12-f864-47bb-825f-5e17f9b2f2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:30.704084 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db7c12-f864-47bb-825f-5e17f9b2f2cb None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:30.704366 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db7c12-f864-47bb-825f-5e17f9b2f2cb None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:30.704634 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db7c12-f864-47bb-825f-5e17f9b2f2cb None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:30.705323 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2db7c12-f864-47bb-825f-5e17f9b2f2cb None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:30.705624 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2db7c12-f864-47bb-825f-5e17f9b2f2cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:30.707014 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2db7c12-f864-47bb-825f-5e17f9b2f2cb None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:30.718898 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 347/708] 10.222.0.50 () {56 vars in 1124 bytes} [Fri May 27 15:01:30 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:30.718898 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2db7c12-f864-47bb-825f-5e17f9b2f2cb None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:30.730062 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e9d4b-ce6c-4747-8d5a-41b0bc392305 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:30.775422 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff8c029-9469-4ff7-9270-f5219e66abc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:30.776224 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8c029-9469-4ff7-9270-f5219e66abc5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:30.776568 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8c029-9469-4ff7-9270-f5219e66abc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:30.776906 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff8c029-9469-4ff7-9270-f5219e66abc5 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:30.777706 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff8c029-9469-4ff7-9270-f5219e66abc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:30.778123 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff8c029-9469-4ff7-9270-f5219e66abc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:30.779619 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff8c029-9469-4ff7-9270-f5219e66abc5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:30.793917 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 362/709] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:30.795528 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f8c83-d700-4d5d-a7ff-d178be72b3de None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:30.796461 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f8c83-d700-4d5d-a7ff-d178be72b3de None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:30.796884 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f8c83-d700-4d5d-a7ff-d178be72b3de None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:30.799081 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 363/710] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:30.809180 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147fd34-5589-4517-8ace-7681119a0c50 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:30.810008 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147fd34-5589-4517-8ace-7681119a0c50 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:30.810475 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147fd34-5589-4517-8ace-7681119a0c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:30.826031 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2db7c12-f864-47bb-825f-5e17f9b2f2cb None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:30.835233 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0147fd34-5589-4517-8ace-7681119a0c50 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:30.836811 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0147fd34-5589-4517-8ace-7681119a0c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:30.861793 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:30.869570 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff7ce86-85f5-480f-8e05-de25dd128f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:30.870375 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff7ce86-85f5-480f-8e05-de25dd128f46 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:30.870733 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff7ce86-85f5-480f-8e05-de25dd128f46 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:30.871119 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff7ce86-85f5-480f-8e05-de25dd128f46 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:30.871926 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff7ce86-85f5-480f-8e05-de25dd128f46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:30.872299 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff7ce86-85f5-480f-8e05-de25dd128f46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:30.873784 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff7ce86-85f5-480f-8e05-de25dd128f46 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:30.891220 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 348/711] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:30.892302 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff7ce86-85f5-480f-8e05-de25dd128f46 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:30.937688 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 364/712] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:01:30.942868 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0147fd34-5589-4517-8ace-7681119a0c50 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:30.981278 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:31.029269 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff7ce86-85f5-480f-8e05-de25dd128f46 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:31.053361 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0147fd34-5589-4517-8ace-7681119a0c50 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:31.069805 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ba6a0a-36ec-48e9-b4e4-a54c01e0f5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:31.070838 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba6a0a-36ec-48e9-b4e4-a54c01e0f5e7 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:31.071147 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba6a0a-36ec-48e9-b4e4-a54c01e0f5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:31.071559 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba6a0a-36ec-48e9-b4e4-a54c01e0f5e7 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:31.072393 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ba6a0a-36ec-48e9-b4e4-a54c01e0f5e7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:31.074291 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ba6a0a-36ec-48e9-b4e4-a54c01e0f5e7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:31.093840 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cf9fc5-9908-43ed-922e-4ade19d38aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:31.094661 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf9fc5-9908-43ed-922e-4ade19d38aba None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:31.094987 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf9fc5-9908-43ed-922e-4ade19d38aba None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:31.095262 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf9fc5-9908-43ed-922e-4ade19d38aba None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:31.101713 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 349/713] 10.222.0.50 () {60 vars in 1132 bytes} [Fri May 27 15:01:30 2022] POST /identity/v3/users => generated 254 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:01:31.102926 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ba6a0a-36ec-48e9-b4e4-a54c01e0f5e7 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:31.103112 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cf9fc5-9908-43ed-922e-4ade19d38aba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:31.104673 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cf9fc5-9908-43ed-922e-4ade19d38aba None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:31.120592 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-83cf9fc5-9908-43ed-922e-4ade19d38aba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:31.120592 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:31.120592 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:31.120592 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:31.120592 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:31.120592 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:31.120592 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:31.120592 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:31.120592 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:31.120592 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:31.120592 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:31.120592 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:31.120592 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 15:01:31.120592 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 15:01:31.120592 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 15:01:31.121681 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 15:01:31.121681 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:31.121681 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:31.121681 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:31.121681 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:31.121681 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:31.121681 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:31.121681 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:31.121681 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:31.121681 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:31.121681 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:31.121681 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:31.121681 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:31.121681 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:31.121681 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:31.121681 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:31.121681 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:31.123033 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 15:01:31.123033 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 15:01:31.123033 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 15:01:31.123033 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 15:01:31.123033 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 15:01:31.123033 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 15:01:31.123033 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:31.123033 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:31.123033 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 365/714] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:01:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:31.128500 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cf9fc5-9908-43ed-922e-4ade19d38aba None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:31.157699 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:31.244727 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ba6a0a-36ec-48e9-b4e4-a54c01e0f5e7 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:31.258322 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:31.287283 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9db75f-feda-49fc-8789-797a1b5b6767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:31.287522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cf9fc5-9908-43ed-922e-4ade19d38aba None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:31.288603 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9db75f-feda-49fc-8789-797a1b5b6767 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:31.289294 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9db75f-feda-49fc-8789-797a1b5b6767 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:31.289771 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9db75f-feda-49fc-8789-797a1b5b6767 None admin] PATH_INFO: `/v3/services/network` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:31.290630 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9db75f-feda-49fc-8789-797a1b5b6767 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:31.292317 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9db75f-feda-49fc-8789-797a1b5b6767 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:31.308498 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-ff9db75f-feda-49fc-8789-797a1b5b6767 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 27 15:01:31.308498 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:31.308498 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 15:01:31.308498 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 15:01:31.308498 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 15:01:31.308498 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 15:01:31.308498 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 15:01:31.308498 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:01:31.308498 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 15:01:31.308498 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:31.308498 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 15:01:31.308498 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:31.308498 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:31.308498 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:31.308498 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:31.308498 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:31.308498 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:31.309578 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:31.309578 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:31.309578 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:31.309578 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:31.309578 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:31.309578 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:31.309578 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 15:01:31.309578 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 15:01:31.309578 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 15:01:31.309578 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 15:01:31.309578 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:31.309578 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:31.309578 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:31.309578 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:31.309578 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:31.309578 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:31.310452 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:31.310452 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:31.310452 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:31.310452 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:31.310452 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:31.310452 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:31.310452 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:31.310452 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:31.310452 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:31.310452 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:31.310452 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 15:01:31.310452 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:01:31.310452 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 15:01:31.310452 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:31.314620 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 350/715] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:01:31 2022] GET /identity/v3/services/network => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:31.320748 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9db75f-feda-49fc-8789-797a1b5b6767 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:31.354651 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bef682-d4d5-42b1-89db-12cf9531b4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:31.355385 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bef682-d4d5-42b1-89db-12cf9531b4e6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:31.355783 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bef682-d4d5-42b1-89db-12cf9531b4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:31.356066 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bef682-d4d5-42b1-89db-12cf9531b4e6 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:31.356990 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bef682-d4d5-42b1-89db-12cf9531b4e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:31.357301 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bef682-d4d5-42b1-89db-12cf9531b4e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:31.358879 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bef682-d4d5-42b1-89db-12cf9531b4e6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:31.371853 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 366/716] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 15:01:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:31.377952 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bef682-d4d5-42b1-89db-12cf9531b4e6 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:31.465841 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:31.475500 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9db75f-feda-49fc-8789-797a1b5b6767 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:31.504296 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bef682-d4d5-42b1-89db-12cf9531b4e6 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:31.523799 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3107bc-43a3-4212-a10b-933713a706d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:31.524860 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3107bc-43a3-4212-a10b-933713a706d5 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:31.525149 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3107bc-43a3-4212-a10b-933713a706d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:31.525428 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3107bc-43a3-4212-a10b-933713a706d5 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:31.526140 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3107bc-43a3-4212-a10b-933713a706d5 None admin] RBAC: Adding query filter params (name=network) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:31.526448 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3107bc-43a3-4212-a10b-933713a706d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:31.528401 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3107bc-43a3-4212-a10b-933713a706d5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:31.538947 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 351/717] 10.222.0.50 () {56 vars in 1112 bytes} [Fri May 27 15:01:31 2022] GET /identity/v3/services?name=network => generated 124 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:31.541436 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3107bc-43a3-4212-a10b-933713a706d5 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:31.555253 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd1b672-0275-4946-989b-c50074cf5850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:31.555923 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd1b672-0275-4946-989b-c50074cf5850 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:31.556207 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd1b672-0275-4946-989b-c50074cf5850 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:31.556483 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd1b672-0275-4946-989b-c50074cf5850 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:31.557320 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd1b672-0275-4946-989b-c50074cf5850 None admin] RBAC: Adding query filter params (role.id=759d845eb6284e42931b4265af448ebc, scope.project.id=daf12585428b41a2b45bdbc85413a6c4, user.id=451a9232f95e494683d81b6078a318d0) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:31.557630 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd1b672-0275-4946-989b-c50074cf5850 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:31.559084 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd1b672-0275-4946-989b-c50074cf5850 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:31.567003 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 367/718] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:01:31 2022] GET /identity/v3/role_assignments?user.id=451a9232f95e494683d81b6078a318d0&scope.project.id=daf12585428b41a2b45bdbc85413a6c4&role.id=759d845eb6284e42931b4265af448ebc => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:01:31.604206 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:31.701057 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3107bc-43a3-4212-a10b-933713a706d5 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:31.737885 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b81773-ad40-4922-8966-83c727a5f429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:31.738567 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b81773-ad40-4922-8966-83c727a5f429 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:31.738870 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b81773-ad40-4922-8966-83c727a5f429 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:31.739155 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b81773-ad40-4922-8966-83c727a5f429 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:31.739876 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b81773-ad40-4922-8966-83c727a5f429 None admin] RBAC: Adding query filter params (type=network) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:31.740191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b81773-ad40-4922-8966-83c727a5f429 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:31.741788 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b81773-ad40-4922-8966-83c727a5f429 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:31.747366 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 352/719] 10.222.0.50 () {56 vars in 1112 bytes} [Fri May 27 15:01:31 2022] GET /identity/v3/services?type=network => generated 124 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:32.059101 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b27b12-625c-4632-99bf-063c30e29f70 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:32.059514 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b27b12-625c-4632-99bf-063c30e29f70 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:32.059689 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b27b12-625c-4632-99bf-063c30e29f70 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:32.061242 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 368/720] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:32.066628 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b849ec6f-9a80-421a-825d-a112a1dcdb95 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:32.066943 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b849ec6f-9a80-421a-825d-a112a1dcdb95 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:32.067227 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b849ec6f-9a80-421a-825d-a112a1dcdb95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:32.086154 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b849ec6f-9a80-421a-825d-a112a1dcdb95 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:32.087359 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b849ec6f-9a80-421a-825d-a112a1dcdb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:32.185239 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 353/721] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:01:32.228293 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b27b12-625c-4632-99bf-063c30e29f70 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:32.354199 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b27b12-625c-4632-99bf-063c30e29f70 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:32.390406 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaaa736-db05-4a4a-aeff-e4c04f0ab855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:32.391591 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaaa736-db05-4a4a-aeff-e4c04f0ab855 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:32.392131 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaaa736-db05-4a4a-aeff-e4c04f0ab855 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:32.392621 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaaa736-db05-4a4a-aeff-e4c04f0ab855 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:32.394664 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:32.404627 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaaa736-db05-4a4a-aeff-e4c04f0ab855 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:01:32.416993 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-8aaaa736-db05-4a4a-aeff-e4c04f0ab855 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:01:32.416993 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:32.416993 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:32.416993 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:32.416993 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:32.416993 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:32.416993 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:32.416993 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:32.416993 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:32.416993 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:32.416993 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:32.416993 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:32.416993 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:01:32.416993 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:01:32.416993 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:01:32.416993 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:01:32.418007 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:01:32.418007 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:01:32.418007 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:01:32.418007 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:01:32.418007 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:32.418007 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:32.418007 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:32.418007 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:32.418007 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:32.418007 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:32.418007 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:32.418007 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:32.418007 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:32.418007 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:32.418007 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:32.418007 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:32.418914 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:32.418914 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:32.418914 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:32.418914 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:32.418914 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:01:32.418914 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:01:32.418914 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:01:32.418914 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:01:32.418914 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:01:32.418914 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:01:32.418914 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:01:32.418914 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:32.422128 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 369/722] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 15:01:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:32.429178 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b849ec6f-9a80-421a-825d-a112a1dcdb95 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:32.479262 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:32.564807 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b849ec6f-9a80-421a-825d-a112a1dcdb95 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:32.602353 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab35ce9b-84c4-478a-82cb-0979f5798ce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:32.603056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35ce9b-84c4-478a-82cb-0979f5798ce8 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:32.603345 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35ce9b-84c4-478a-82cb-0979f5798ce8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:32.603620 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35ce9b-84c4-478a-82cb-0979f5798ce8 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:32.604497 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab35ce9b-84c4-478a-82cb-0979f5798ce8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:32.604755 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab35ce9b-84c4-478a-82cb-0979f5798ce8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:32.606132 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab35ce9b-84c4-478a-82cb-0979f5798ce8 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:32.618770 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 354/723] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:01:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:32.627938 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaaa736-db05-4a4a-aeff-e4c04f0ab855 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:32.739234 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaaa736-db05-4a4a-aeff-e4c04f0ab855 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:32.773817 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2db4588-d602-42d9-84ed-7741517deb6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:32.775009 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db4588-d602-42d9-84ed-7741517deb6e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:32.775498 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db4588-d602-42d9-84ed-7741517deb6e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:32.775988 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db4588-d602-42d9-84ed-7741517deb6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:32.787065 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2db4588-d602-42d9-84ed-7741517deb6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:32.788783 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2db4588-d602-42d9-84ed-7741517deb6e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:32.806481 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-b2db4588-d602-42d9-84ed-7741517deb6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:32.806481 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:32.806481 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:32.806481 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:32.806481 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:32.806481 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:32.806481 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:32.806481 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:32.806481 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:32.806481 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:32.806481 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:32.806481 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:32.806481 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:32.806481 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:32.806481 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:32.807447 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:32.807447 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:32.807447 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:32.807447 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:32.807447 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:32.807447 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:32.807447 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:32.807447 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:32.807447 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:32.807447 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:32.807447 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:32.807447 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:32.807447 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:32.807447 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:32.807447 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:32.807447 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:32.807447 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:32.808433 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:32.808433 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:32.808433 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:32.808433 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:32.810725 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 370/724] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:32.819954 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab35ce9b-84c4-478a-82cb-0979f5798ce8 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:32.982293 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:32.991324 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab35ce9b-84c4-478a-82cb-0979f5798ce8 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:33.038984 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab2f1c3-182d-447a-86e0-dd07e668a8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:33.039657 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2f1c3-182d-447a-86e0-dd07e668a8b8 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:33.039940 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2f1c3-182d-447a-86e0-dd07e668a8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:33.040217 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab2f1c3-182d-447a-86e0-dd07e668a8b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:33.040951 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab2f1c3-182d-447a-86e0-dd07e668a8b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:33.041260 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab2f1c3-182d-447a-86e0-dd07e668a8b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:33.042633 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab2f1c3-182d-447a-86e0-dd07e668a8b8 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:33.056557 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 355/725] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:33.062665 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2db4588-d602-42d9-84ed-7741517deb6e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:33.118991 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:33.184489 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2db4588-d602-42d9-84ed-7741517deb6e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:33.219952 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a8601-f921-4626-b986-094c193dd8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:33.220752 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a8601-f921-4626-b986-094c193dd8cc None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:33.221032 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a8601-f921-4626-b986-094c193dd8cc None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:33.221302 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a8601-f921-4626-b986-094c193dd8cc None admin] PATH_INFO: `/v3/users/swift` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:33.228516 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174a8601-f921-4626-b986-094c193dd8cc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:33.229957 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174a8601-f921-4626-b986-094c193dd8cc None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:33.245439 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-174a8601-f921-4626-b986-094c193dd8cc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 15:01:33.245439 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:33.245439 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:33.245439 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:33.245439 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:33.245439 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:33.245439 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:33.245439 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:33.245439 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:33.245439 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:33.245439 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:33.245439 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:33.245439 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 15:01:33.245439 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 15:01:33.245439 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 15:01:33.245439 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 15:01:33.246431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:33.246431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:33.246431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 15:01:33.246431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:33.246431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 15:01:33.246431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:33.246431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:33.246431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:33.246431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:33.246431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:33.246431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:33.246431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:33.246431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:33.246431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:33.246431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:33.246431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:33.246431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:33.246431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:33.247411 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:33.247411 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:33.247411 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 15:01:33.247411 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 15:01:33.247411 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 15:01:33.247411 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 15:01:33.247411 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 15:01:33.247411 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 15:01:33.247411 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 15:01:33.247411 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:33.247411 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 371/726] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 15:01:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:33.254906 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab2f1c3-182d-447a-86e0-dd07e668a8b8 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:33.396697 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab2f1c3-182d-447a-86e0-dd07e668a8b8 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:33.442382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8725db-950d-425c-b945-8eb4c5be2ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:33.443160 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8725db-950d-425c-b945-8eb4c5be2ecb None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:33.443364 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8725db-950d-425c-b945-8eb4c5be2ecb None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:33.443646 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8725db-950d-425c-b945-8eb4c5be2ecb None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:33.444482 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8725db-950d-425c-b945-8eb4c5be2ecb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:33.444795 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8725db-950d-425c-b945-8eb4c5be2ecb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:33.446226 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8725db-950d-425c-b945-8eb4c5be2ecb None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:33.471865 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 356/727] 10.222.0.50 () {56 vars in 1135 bytes} [Fri May 27 15:01:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:33.477236 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a8601-f921-4626-b986-094c193dd8cc None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:33.496539 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e6412-4564-4992-af00-a6a8521adcc0 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:33.496804 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e6412-4564-4992-af00-a6a8521adcc0 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:33.497025 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783e6412-4564-4992-af00-a6a8521adcc0 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:33.498701 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 357/728] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:33.507960 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df856b18-8a5c-478a-9283-21aa2d2c5896 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:33.508250 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df856b18-8a5c-478a-9283-21aa2d2c5896 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:33.508536 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df856b18-8a5c-478a-9283-21aa2d2c5896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:33.536537 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-df856b18-8a5c-478a-9283-21aa2d2c5896 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:33.537725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-df856b18-8a5c-478a-9283-21aa2d2c5896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:33.557740 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:33.613800 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a8601-f921-4626-b986-094c193dd8cc None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:33.627218 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:33.654900 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256f5330-aa05-41a8-bec0-31d07d52238e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:33.656723 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256f5330-aa05-41a8-bec0-31d07d52238e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:33.657007 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256f5330-aa05-41a8-bec0-31d07d52238e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:33.657285 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256f5330-aa05-41a8-bec0-31d07d52238e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:33.741541 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256f5330-aa05-41a8-bec0-31d07d52238e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:33.743075 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256f5330-aa05-41a8-bec0-31d07d52238e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:33.786700 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 358/729] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 15:01:33.864268 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df856b18-8a5c-478a-9283-21aa2d2c5896 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:33.873448 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 372/730] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:01:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 27 15:01:33.880744 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256f5330-aa05-41a8-bec0-31d07d52238e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:33.891215 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:34.019698 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df856b18-8a5c-478a-9283-21aa2d2c5896 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:34.062228 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7046bcbb-0de9-4ab6-a97e-0ff85645d3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:34.062826 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7046bcbb-0de9-4ab6-a97e-0ff85645d3f4 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:34.063171 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7046bcbb-0de9-4ab6-a97e-0ff85645d3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:34.063454 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7046bcbb-0de9-4ab6-a97e-0ff85645d3f4 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:34.064191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7046bcbb-0de9-4ab6-a97e-0ff85645d3f4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:34.065578 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7046bcbb-0de9-4ab6-a97e-0ff85645d3f4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:34.097879 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 359/731] 10.222.0.50 () {60 vars in 1141 bytes} [Fri May 27 15:01:33 2022] POST /identity/v3/services => generated 251 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:01:34.121041 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256f5330-aa05-41a8-bec0-31d07d52238e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:34.212407 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141e54e-2002-43d9-a298-3abcf072298f None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:34.212692 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141e54e-2002-43d9-a298-3abcf072298f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:34.213429 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141e54e-2002-43d9-a298-3abcf072298f None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:34.215402 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 360/732] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 15:01:34.218881 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10697ff5-bea9-4c19-b1d9-512c25eb14d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:34.218881 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10697ff5-bea9-4c19-b1d9-512c25eb14d0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:34.218881 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10697ff5-bea9-4c19-b1d9-512c25eb14d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:34.218881 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10697ff5-bea9-4c19-b1d9-512c25eb14d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:34.221716 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3db58a-9bec-4171-9f11-e4516b45b4c2 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:34.222230 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3db58a-9bec-4171-9f11-e4516b45b4c2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:34.222736 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3db58a-9bec-4171-9f11-e4516b45b4c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:34.231285 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10697ff5-bea9-4c19-b1d9-512c25eb14d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:34.232962 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10697ff5-bea9-4c19-b1d9-512c25eb14d0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:34.240825 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:34.253679 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-10697ff5-bea9-4c19-b1d9-512c25eb14d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:34.253679 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:34.253679 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:34.253679 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:34.253679 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:34.253679 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:34.253679 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:34.253679 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:34.253679 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:34.253679 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:34.253679 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:34.253679 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:34.253679 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:34.253679 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:34.253679 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:34.254638 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:34.254638 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:34.254638 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:34.254638 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:34.254638 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:34.254638 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:34.254638 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:34.254638 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:34.254638 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:34.254638 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:34.254638 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:34.254638 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:34.254638 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:34.254638 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:34.254638 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:34.254638 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:34.254638 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:34.255582 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:34.255582 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:34.255582 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:34.255582 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:34.255582 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 373/733] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:34.263962 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10697ff5-bea9-4c19-b1d9-512c25eb14d0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:34.268833 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-6f3db58a-9bec-4171-9f11-e4516b45b4c2 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:34.271174 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6f3db58a-9bec-4171-9f11-e4516b45b4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:34.392443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:34.392675 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 361/734] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:34 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 27 15:01:34.417194 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3db58a-9bec-4171-9f11-e4516b45b4c2 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:34.423578 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10697ff5-bea9-4c19-b1d9-512c25eb14d0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:34.467681 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fa9a91-6ead-471f-89a2-4a3f9f4c28f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:34.468358 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fa9a91-6ead-471f-89a2-4a3f9f4c28f9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:34.468647 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fa9a91-6ead-471f-89a2-4a3f9f4c28f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:34.468939 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fa9a91-6ead-471f-89a2-4a3f9f4c28f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:34.470095 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fa9a91-6ead-471f-89a2-4a3f9f4c28f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:34.470409 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fa9a91-6ead-471f-89a2-4a3f9f4c28f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:34.472115 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fa9a91-6ead-471f-89a2-4a3f9f4c28f9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:34.490177 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 374/735] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:34.492283 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d741d-2237-44d2-97f7-8757b617ef19 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:34.492830 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d741d-2237-44d2-97f7-8757b617ef19 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:34.493121 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d741d-2237-44d2-97f7-8757b617ef19 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:34.498659 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 375/736] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:34.500229 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48d741d-2237-44d2-97f7-8757b617ef19 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:34.539812 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3db58a-9bec-4171-9f11-e4516b45b4c2 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:34.579256 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:34.588053 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d870217e-a12f-4d0f-b33b-679e806ee7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:34.590434 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870217e-a12f-4d0f-b33b-679e806ee7b1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:34.590952 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870217e-a12f-4d0f-b33b-679e806ee7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:34.591560 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870217e-a12f-4d0f-b33b-679e806ee7b1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:34.604006 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d870217e-a12f-4d0f-b33b-679e806ee7b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:01:34.612991 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-d870217e-a12f-4d0f-b33b-679e806ee7b1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 15:01:34.612991 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:34.612991 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:34.612991 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:34.612991 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:34.612991 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:34.612991 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:34.612991 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:34.612991 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:34.612991 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:34.612991 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:34.612991 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:34.612991 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:01:34.612991 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:01:34.612991 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:01:34.612991 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:01:34.614063 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:01:34.614063 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:01:34.614063 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:01:34.614063 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:01:34.614063 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:34.614063 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:34.614063 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:34.614063 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:34.614063 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:34.614063 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:34.614063 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:34.614063 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:34.614063 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:34.614063 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:34.614063 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:34.614063 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:34.614935 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:34.614935 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:34.614935 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:34.614935 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:34.614935 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:01:34.614935 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:01:34.614935 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:01:34.614935 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:01:34.614935 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:01:34.614935 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:01:34.614935 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 15:01:34.614935 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:34.623578 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 362/737] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 15:01:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:01:34.626098 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df3444-27e5-48e4-85a2-02855277c69c None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:34.626617 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df3444-27e5-48e4-85a2-02855277c69c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:34.627174 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02df3444-27e5-48e4-85a2-02855277c69c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:34.638039 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48d741d-2237-44d2-97f7-8757b617ef19 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:34.646351 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-02df3444-27e5-48e4-85a2-02855277c69c None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:34.647991 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-02df3444-27e5-48e4-85a2-02855277c69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:34.655358 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:34.683650 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee79e77a-0c78-4ac4-a5f7-d36ac3c79e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:34.684531 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79e77a-0c78-4ac4-a5f7-d36ac3c79e19 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:34.684838 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79e77a-0c78-4ac4-a5f7-d36ac3c79e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:34.685118 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79e77a-0c78-4ac4-a5f7-d36ac3c79e19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:34.696644 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee79e77a-0c78-4ac4-a5f7-d36ac3c79e19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:34.698656 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee79e77a-0c78-4ac4-a5f7-d36ac3c79e19 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:34.718094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-ee79e77a-0c78-4ac4-a5f7-d36ac3c79e19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:34.718094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:34.718094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:34.718094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:34.718094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:34.718094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:34.718094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:34.718094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:34.718094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:34.718094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:34.718094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:34.718094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:34.718094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 15:01:34.718094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 15:01:34.718094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 15:01:34.719198 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 15:01:34.719198 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:34.719198 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:34.719198 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:34.719198 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:34.719198 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:34.719198 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:34.719198 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:34.719198 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:34.719198 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:34.719198 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:34.719198 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:34.719198 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:34.719198 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:34.719198 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:34.719198 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:34.719198 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:34.720115 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 15:01:34.720115 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 15:01:34.720115 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 15:01:34.720115 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 15:01:34.720115 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 15:01:34.720115 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 15:01:34.720115 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:34.720115 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:34.720115 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 376/738] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:01:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:34.720606 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee79e77a-0c78-4ac4-a5f7-d36ac3c79e19 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:34.731884 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 363/739] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:34 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:01:34.732803 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02df3444-27e5-48e4-85a2-02855277c69c None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:34.829027 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee79e77a-0c78-4ac4-a5f7-d36ac3c79e19 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:34.844075 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02df3444-27e5-48e4-85a2-02855277c69c None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:34.863953 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbff59e-1665-4fba-afe9-fb5b5df15b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:34.864624 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbff59e-1665-4fba-afe9-fb5b5df15b05 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:34.864953 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbff59e-1665-4fba-afe9-fb5b5df15b05 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:34.865187 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbff59e-1665-4fba-afe9-fb5b5df15b05 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:34.866056 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbff59e-1665-4fba-afe9-fb5b5df15b05 None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:34.866361 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbff59e-1665-4fba-afe9-fb5b5df15b05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:34.867763 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbff59e-1665-4fba-afe9-fb5b5df15b05 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:34.868167 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:34.881453 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc49479-04b8-4a5f-9ba3-be578afd31f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:34.882136 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc49479-04b8-4a5f-9ba3-be578afd31f5 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:34.882644 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc49479-04b8-4a5f-9ba3-be578afd31f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:34.882886 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 377/740] 10.222.0.50 () {56 vars in 1103 bytes} [Fri May 27 15:01:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:34.883020 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc49479-04b8-4a5f-9ba3-be578afd31f5 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:34.883655 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc49479-04b8-4a5f-9ba3-be578afd31f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:34.883963 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc49479-04b8-4a5f-9ba3-be578afd31f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:34.884174 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a18b07-4e8d-4de5-8ac8-a3c138977f1c None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:34.884466 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a18b07-4e8d-4de5-8ac8-a3c138977f1c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:34.884746 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a18b07-4e8d-4de5-8ac8-a3c138977f1c None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:34.885597 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc49479-04b8-4a5f-9ba3-be578afd31f5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:34.890467 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 378/741] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:34.890467 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a18b07-4e8d-4de5-8ac8-a3c138977f1c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:34.902591 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 364/742] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 15:01:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:34.903476 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc49479-04b8-4a5f-9ba3-be578afd31f5 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:34.931768 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:35.013997 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a18b07-4e8d-4de5-8ac8-a3c138977f1c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:35.018721 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc49479-04b8-4a5f-9ba3-be578afd31f5 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:35.053374 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d8a7e0-16fd-490e-a7a5-c3d8f9d3ff6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:35.054401 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf699d6-da24-4779-ac50-76f7e5becc81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:35.054881 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d8a7e0-16fd-490e-a7a5-c3d8f9d3ff6e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:35.055209 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf699d6-da24-4779-ac50-76f7e5becc81 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:35.055486 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf699d6-da24-4779-ac50-76f7e5becc81 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:35.055757 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf699d6-da24-4779-ac50-76f7e5becc81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:35.055968 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d8a7e0-16fd-490e-a7a5-c3d8f9d3ff6e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:35.056467 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d8a7e0-16fd-490e-a7a5-c3d8f9d3ff6e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:35.066900 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf699d6-da24-4779-ac50-76f7e5becc81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:35.068347 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf699d6-da24-4779-ac50-76f7e5becc81 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:35.070356 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d8a7e0-16fd-490e-a7a5-c3d8f9d3ff6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:01:35.083177 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-f9d8a7e0-16fd-490e-a7a5-c3d8f9d3ff6e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:01:35.083177 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:35.083177 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:35.083177 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:35.083177 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:35.083177 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:35.083177 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:35.083177 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:35.083177 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:35.083177 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:35.083177 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:35.083177 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:35.083177 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:01:35.083177 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:01:35.083177 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:01:35.083177 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:01:35.084296 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:01:35.084296 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:01:35.084296 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:01:35.084296 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:01:35.084296 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:35.084296 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:35.084296 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:35.084296 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:35.084296 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:35.084296 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:35.084296 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:35.084296 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:35.084296 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:35.084296 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:35.084296 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:35.084296 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:35.085351 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:35.085351 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:35.085351 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:35.085351 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:35.085351 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:01:35.085351 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:01:35.085351 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:01:35.085351 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:01:35.085351 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:01:35.085351 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:01:35.085351 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:01:35.085351 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:35.086400 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-1cf699d6-da24-4779-ac50-76f7e5becc81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:35.086400 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:35.086400 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:35.086400 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:35.086400 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:35.086400 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:35.086400 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:35.086400 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:35.086400 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:35.086400 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:35.086400 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:35.086400 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:35.086400 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:35.086400 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:35.086400 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:35.087447 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:35.087447 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:35.087447 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:35.087447 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:35.087447 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:35.087447 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:35.087447 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:35.087447 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:35.087447 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:35.087447 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:35.087447 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:35.087447 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:35.087447 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:35.087447 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:35.087447 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:35.087447 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:35.087447 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:35.088634 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:35.088634 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:35.088634 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:35.088634 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:35.088634 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 365/743] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:35.089066 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c725ceac-b4dc-47a9-8cde-579edb4b6c85 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:35.089353 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c725ceac-b4dc-47a9-8cde-579edb4b6c85 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:35.089632 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c725ceac-b4dc-47a9-8cde-579edb4b6c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:35.090399 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 379/744] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 15:01:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:35.091518 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d8a7e0-16fd-490e-a7a5-c3d8f9d3ff6e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:35.114342 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c725ceac-b4dc-47a9-8cde-579edb4b6c85 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:35.115575 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c725ceac-b4dc-47a9-8cde-579edb4b6c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:35.142674 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:35.193051 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 366/745] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:35 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 27 15:01:35.194205 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c725ceac-b4dc-47a9-8cde-579edb4b6c85 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:35.213943 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:35.222067 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d8a7e0-16fd-490e-a7a5-c3d8f9d3ff6e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:35.257996 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d5152-6b12-43dc-8011-96409ccae138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:35.258650 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d5152-6b12-43dc-8011-96409ccae138 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:35.258968 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d5152-6b12-43dc-8011-96409ccae138 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:35.259248 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d5152-6b12-43dc-8011-96409ccae138 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:35.260068 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2d5152-6b12-43dc-8011-96409ccae138 None admin] RBAC: Adding query filter params (role.id=759d845eb6284e42931b4265af448ebc, scope.project.id=daf12585428b41a2b45bdbc85413a6c4, user.id=18903f2eab7c415096e075d47f0b9a4d) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:35.260387 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2d5152-6b12-43dc-8011-96409ccae138 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:35.261806 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2d5152-6b12-43dc-8011-96409ccae138 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:35.267844 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 380/746] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:01:35 2022] GET /identity/v3/role_assignments?user.id=18903f2eab7c415096e075d47f0b9a4d&scope.project.id=daf12585428b41a2b45bdbc85413a6c4&role.id=759d845eb6284e42931b4265af448ebc => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:35.268629 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d5152-6b12-43dc-8011-96409ccae138 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:35.344767 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c725ceac-b4dc-47a9-8cde-579edb4b6c85 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:35.398574 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9991ebe-170f-4087-9003-c5ec9b297298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:35.399290 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9991ebe-170f-4087-9003-c5ec9b297298 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:35.399576 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9991ebe-170f-4087-9003-c5ec9b297298 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:35.399854 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9991ebe-170f-4087-9003-c5ec9b297298 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:35.401282 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9991ebe-170f-4087-9003-c5ec9b297298 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:35.401612 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9991ebe-170f-4087-9003-c5ec9b297298 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:35.403377 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9991ebe-170f-4087-9003-c5ec9b297298 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:35.415528 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d5152-6b12-43dc-8011-96409ccae138 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:35.423055 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 367/747] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:35.423995 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9991ebe-170f-4087-9003-c5ec9b297298 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:35.454822 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:35.476730 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef9d0a0-e66b-4d4d-9227-b64611d55d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:35.477600 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef9d0a0-e66b-4d4d-9227-b64611d55d8e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:35.478011 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef9d0a0-e66b-4d4d-9227-b64611d55d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:35.478299 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef9d0a0-e66b-4d4d-9227-b64611d55d8e None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:35.479118 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef9d0a0-e66b-4d4d-9227-b64611d55d8e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:35.479696 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef9d0a0-e66b-4d4d-9227-b64611d55d8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:35.481454 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef9d0a0-e66b-4d4d-9227-b64611d55d8e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:35.493577 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 381/748] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:01:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:35.494425 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef9d0a0-e66b-4d4d-9227-b64611d55d8e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:35.528041 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:35.545149 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9991ebe-170f-4087-9003-c5ec9b297298 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:35.589241 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133768ef-f5bd-4536-aedf-17b9f54fe12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:35.590306 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133768ef-f5bd-4536-aedf-17b9f54fe12b None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:35.590814 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133768ef-f5bd-4536-aedf-17b9f54fe12b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:35.591320 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133768ef-f5bd-4536-aedf-17b9f54fe12b None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:35.592244 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133768ef-f5bd-4536-aedf-17b9f54fe12b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:35.593896 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133768ef-f5bd-4536-aedf-17b9f54fe12b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:35.612978 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 368/749] 10.222.0.50 () {60 vars in 1141 bytes} [Fri May 27 15:01:35 2022] POST /identity/v3/services => generated 240 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:01:35.614164 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133768ef-f5bd-4536-aedf-17b9f54fe12b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:35.642275 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef9d0a0-e66b-4d4d-9227-b64611d55d8e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:35.738104 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d3820a-014f-4cda-a100-29c6f25d59c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:35.738840 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d3820a-014f-4cda-a100-29c6f25d59c5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:35.739136 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d3820a-014f-4cda-a100-29c6f25d59c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:35.739406 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d3820a-014f-4cda-a100-29c6f25d59c5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:35.757770 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d3820a-014f-4cda-a100-29c6f25d59c5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:35.761518 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d3820a-014f-4cda-a100-29c6f25d59c5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:35.793578 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-f2d3820a-014f-4cda-a100-29c6f25d59c5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 15:01:35.793578 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:35.793578 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:35.793578 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:35.793578 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:35.793578 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:35.793578 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:35.793578 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:35.793578 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:35.793578 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:35.793578 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:35.793578 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:35.793578 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 15:01:35.793578 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 15:01:35.793578 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 15:01:35.793578 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 15:01:35.794623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:35.794623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:35.794623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 15:01:35.794623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:35.794623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 15:01:35.794623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:35.794623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:35.794623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:35.794623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:35.794623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:35.794623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:35.794623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:35.794623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:35.794623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:35.794623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:35.794623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:35.794623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:35.794623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:35.796203 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:35.796203 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:35.796203 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 15:01:35.796203 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 15:01:35.796203 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 15:01:35.796203 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 15:01:35.796203 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 15:01:35.796203 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 15:01:35.796203 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 15:01:35.796203 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:35.803965 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 382/750] 10.222.0.50 () {56 vars in 1117 bytes} [Fri May 27 15:01:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:35.809437 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d3820a-014f-4cda-a100-29c6f25d59c5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:35.814918 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:35.839868 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133768ef-f5bd-4536-aedf-17b9f54fe12b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:35.884042 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d4e6d-2f5a-4e4d-a007-49e5a3814c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:35.884841 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d4e6d-2f5a-4e4d-a007-49e5a3814c7d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:35.885122 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d4e6d-2f5a-4e4d-a007-49e5a3814c7d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:35.885400 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d4e6d-2f5a-4e4d-a007-49e5a3814c7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:35.899031 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78d4e6d-2f5a-4e4d-a007-49e5a3814c7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:35.901227 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78d4e6d-2f5a-4e4d-a007-49e5a3814c7d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:35.910701 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:35.924932 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-f78d4e6d-2f5a-4e4d-a007-49e5a3814c7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:35.924932 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:35.924932 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:35.924932 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:35.924932 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:35.924932 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:35.924932 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:35.924932 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:35.924932 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:35.924932 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:35.924932 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:35.924932 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:35.924932 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:35.924932 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:35.924932 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:35.925884 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:35.925884 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:35.925884 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:35.925884 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:35.925884 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:35.925884 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:35.925884 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:35.925884 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:35.925884 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:35.925884 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:35.925884 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:35.925884 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:35.925884 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:35.925884 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:35.925884 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:35.925884 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:35.925884 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:35.926817 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:35.926817 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:35.926817 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:35.926817 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:35.926817 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 369/751] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:35.932331 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d4e6d-2f5a-4e4d-a007-49e5a3814c7d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:35.943350 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d3820a-014f-4cda-a100-29c6f25d59c5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:35.987235 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4502a479-2112-418b-b0ae-2a519720289e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:35.988312 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4502a479-2112-418b-b0ae-2a519720289e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:35.988823 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4502a479-2112-418b-b0ae-2a519720289e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:35.989325 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4502a479-2112-418b-b0ae-2a519720289e None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:35.990380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4502a479-2112-418b-b0ae-2a519720289e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:35.990921 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4502a479-2112-418b-b0ae-2a519720289e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:35.992546 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4502a479-2112-418b-b0ae-2a519720289e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:36.011536 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 383/752] 10.222.0.50 () {56 vars in 1149 bytes} [Fri May 27 15:01:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:36.018359 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4502a479-2112-418b-b0ae-2a519720289e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:36.039765 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d4e6d-2f5a-4e4d-a007-49e5a3814c7d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:36.075558 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9348f009-8afc-4c9a-85ca-c0383dcb7a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:36.076225 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9348f009-8afc-4c9a-85ca-c0383dcb7a65 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:36.076507 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9348f009-8afc-4c9a-85ca-c0383dcb7a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:36.076830 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9348f009-8afc-4c9a-85ca-c0383dcb7a65 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:36.078253 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9348f009-8afc-4c9a-85ca-c0383dcb7a65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:36.078558 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9348f009-8afc-4c9a-85ca-c0383dcb7a65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:36.080318 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9348f009-8afc-4c9a-85ca-c0383dcb7a65 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:36.082691 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:36.096351 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 370/753] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:01:36.102124 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9348f009-8afc-4c9a-85ca-c0383dcb7a65 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:36.140244 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4502a479-2112-418b-b0ae-2a519720289e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:36.177825 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c768591-21a1-4d25-a589-7bcf6000e43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:36.178722 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c768591-21a1-4d25-a589-7bcf6000e43f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:36.179200 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c768591-21a1-4d25-a589-7bcf6000e43f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:36.179638 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c768591-21a1-4d25-a589-7bcf6000e43f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:36.183665 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:36.235598 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9348f009-8afc-4c9a-85ca-c0383dcb7a65 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:36.253061 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c768591-21a1-4d25-a589-7bcf6000e43f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:36.254549 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c768591-21a1-4d25-a589-7bcf6000e43f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:36.300381 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445c5ef6-5080-4fe8-a5f8-163e46666996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:36.301621 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c5ef6-5080-4fe8-a5f8-163e46666996 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:36.302153 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c5ef6-5080-4fe8-a5f8-163e46666996 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:36.302624 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c5ef6-5080-4fe8-a5f8-163e46666996 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:36.318686 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445c5ef6-5080-4fe8-a5f8-163e46666996 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:36.320603 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445c5ef6-5080-4fe8-a5f8-163e46666996 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:36.357003 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-445c5ef6-5080-4fe8-a5f8-163e46666996 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 15:01:36.357003 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:36.357003 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:36.357003 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:36.357003 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:36.357003 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:36.357003 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:36.357003 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:36.357003 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:36.357003 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:36.357003 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:36.357003 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:36.357003 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 15:01:36.357003 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 15:01:36.357003 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 15:01:36.357003 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 15:01:36.358743 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:36.358743 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:36.358743 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 15:01:36.358743 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:36.358743 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 15:01:36.358743 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:36.358743 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:36.358743 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:36.358743 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:36.358743 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:36.358743 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:36.358743 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:36.358743 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:36.358743 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:36.358743 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:36.358743 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:36.358743 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:36.358743 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:36.360010 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:36.360010 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:36.360010 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 15:01:36.360010 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 15:01:36.360010 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 15:01:36.360010 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 15:01:36.360010 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 15:01:36.360010 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 15:01:36.360010 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 15:01:36.360010 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:36.360668 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 384/754] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:01:36 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 15:01:36.361088 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 371/755] 10.222.0.50 () {56 vars in 1093 bytes} [Fri May 27 15:01:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:36.368834 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c768591-21a1-4d25-a589-7bcf6000e43f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:36.369029 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445c5ef6-5080-4fe8-a5f8-163e46666996 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:36.440201 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:36.479239 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:36.510893 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c768591-21a1-4d25-a589-7bcf6000e43f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:36.552937 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad773c7d-781f-4556-ba2c-86f37c1f83e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:36.553797 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad773c7d-781f-4556-ba2c-86f37c1f83e3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:36.554026 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad773c7d-781f-4556-ba2c-86f37c1f83e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:36.554308 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad773c7d-781f-4556-ba2c-86f37c1f83e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:36.574151 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad773c7d-781f-4556-ba2c-86f37c1f83e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:36.577852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad773c7d-781f-4556-ba2c-86f37c1f83e3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:36.582177 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445c5ef6-5080-4fe8-a5f8-163e46666996 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:36.609623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-ad773c7d-781f-4556-ba2c-86f37c1f83e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:36.609623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:36.609623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:36.609623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:36.609623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:36.609623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:36.609623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:36.609623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:36.609623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:36.609623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:36.609623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:36.609623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:36.609623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:36.609623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:36.609623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:36.610575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:36.610575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:36.610575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:36.610575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:36.610575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:36.610575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:36.610575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:36.610575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:36.610575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:36.610575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:36.610575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:36.610575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:36.610575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:36.610575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:36.610575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:36.610575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:36.610575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:36.611630 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:36.611630 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:36.611630 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:36.611630 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:36.611861 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 385/756] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:36.619845 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad773c7d-781f-4556-ba2c-86f37c1f83e3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:36.641579 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb8889d-db7b-47b6-b19d-e3910b8eefae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:36.642423 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb8889d-db7b-47b6-b19d-e3910b8eefae None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:36.642781 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb8889d-db7b-47b6-b19d-e3910b8eefae None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:36.643162 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb8889d-db7b-47b6-b19d-e3910b8eefae None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:36.644067 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb8889d-db7b-47b6-b19d-e3910b8eefae None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:36.644447 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb8889d-db7b-47b6-b19d-e3910b8eefae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:36.646003 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb8889d-db7b-47b6-b19d-e3910b8eefae None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:36.670368 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 372/757] 10.222.0.50 () {56 vars in 1133 bytes} [Fri May 27 15:01:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:01:36.670368 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e3b9fa-580a-4651-9d43-26aa3c95aefa None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:36.670368 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e3b9fa-580a-4651-9d43-26aa3c95aefa None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:36.670368 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e3b9fa-580a-4651-9d43-26aa3c95aefa None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:36.670368 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 373/758] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:36.670368 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e3b9fa-580a-4651-9d43-26aa3c95aefa None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:36.732638 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:36.746540 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad773c7d-781f-4556-ba2c-86f37c1f83e3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:36.782163 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e874a1-2ab1-4a1b-87d8-0c0a76436e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:36.782837 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e874a1-2ab1-4a1b-87d8-0c0a76436e3e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:36.783172 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e874a1-2ab1-4a1b-87d8-0c0a76436e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:36.783453 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e874a1-2ab1-4a1b-87d8-0c0a76436e3e None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:36.784177 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e874a1-2ab1-4a1b-87d8-0c0a76436e3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:36.784480 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e874a1-2ab1-4a1b-87d8-0c0a76436e3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:36.785840 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e874a1-2ab1-4a1b-87d8-0c0a76436e3e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:36.788531 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e3b9fa-580a-4651-9d43-26aa3c95aefa None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:36.802687 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 386/759] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:36.802687 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c2d58-0b5c-41ab-87d6-648a30c95fe4 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:36.802687 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c2d58-0b5c-41ab-87d6-648a30c95fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:36.802687 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c2d58-0b5c-41ab-87d6-648a30c95fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:36.809931 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:36.820978 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1c8c2d58-0b5c-41ab-87d6-648a30c95fe4 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:36.822289 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1c8c2d58-0b5c-41ab-87d6-648a30c95fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:36.833010 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3239dd4-9ac4-40d5-af78-b6d72ce4511a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:36.834192 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3239dd4-9ac4-40d5-af78-b6d72ce4511a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:36.834682 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3239dd4-9ac4-40d5-af78-b6d72ce4511a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:36.835209 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3239dd4-9ac4-40d5-af78-b6d72ce4511a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:36.847425 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3239dd4-9ac4-40d5-af78-b6d72ce4511a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:36.849493 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3239dd4-9ac4-40d5-af78-b6d72ce4511a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:36.869558 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-f3239dd4-9ac4-40d5-af78-b6d72ce4511a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:36.869558 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:36.869558 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:36.869558 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:36.869558 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:36.869558 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:36.869558 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:36.869558 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:36.869558 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:36.869558 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:36.869558 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:36.869558 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:36.869558 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:36.869558 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:36.869558 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:36.870485 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:36.870485 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:36.870485 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:36.870485 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:36.870485 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:36.870485 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:36.870485 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:36.870485 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:36.870485 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:36.870485 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:36.870485 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:36.870485 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:36.870485 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:36.870485 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:36.870485 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:36.870485 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:36.870485 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:36.871531 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:36.871531 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:36.871531 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:36.871531 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:36.878158 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 374/760] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:36.879268 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3239dd4-9ac4-40d5-af78-b6d72ce4511a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:36.902872 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 387/761] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:01:36.902872 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c2d58-0b5c-41ab-87d6-648a30c95fe4 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:36.985415 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:37.010728 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c2d58-0b5c-41ab-87d6-648a30c95fe4 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:37.011738 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3239dd4-9ac4-40d5-af78-b6d72ce4511a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:37.048284 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aefd402-83dc-40da-a85c-c76b29d14088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:37.048785 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1717df60-e144-4dfa-b483-d7a6c9c926af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:37.049506 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aefd402-83dc-40da-a85c-c76b29d14088 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:37.049637 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1717df60-e144-4dfa-b483-d7a6c9c926af None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:37.050036 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1717df60-e144-4dfa-b483-d7a6c9c926af None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:37.050203 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aefd402-83dc-40da-a85c-c76b29d14088 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:37.050495 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1717df60-e144-4dfa-b483-d7a6c9c926af None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:37.050770 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aefd402-83dc-40da-a85c-c76b29d14088 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:37.051455 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1717df60-e144-4dfa-b483-d7a6c9c926af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:37.051829 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1717df60-e144-4dfa-b483-d7a6c9c926af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:37.053401 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1717df60-e144-4dfa-b483-d7a6c9c926af None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:37.058337 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aefd402-83dc-40da-a85c-c76b29d14088 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:37.059982 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aefd402-83dc-40da-a85c-c76b29d14088 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:37.070062 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 388/762] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:37.070062 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1717df60-e144-4dfa-b483-d7a6c9c926af None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:37.079812 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-3aefd402-83dc-40da-a85c-c76b29d14088 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:37.079812 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:37.079812 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:37.079812 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:37.079812 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:37.079812 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:37.079812 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:37.079812 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:37.079812 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:37.079812 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:37.079812 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:37.079812 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:37.079812 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 15:01:37.079812 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 15:01:37.079812 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 15:01:37.080725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 15:01:37.080725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:37.080725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:37.080725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:37.080725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:37.080725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:37.080725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:37.080725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:37.080725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:37.080725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:37.080725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:37.080725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:37.080725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:37.080725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:37.080725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:37.080725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:37.080725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:37.081637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 15:01:37.081637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 15:01:37.081637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 15:01:37.081637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 15:01:37.081637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 15:01:37.081637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 15:01:37.081637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:37.081637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:37.083962 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 375/763] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:01:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:37.084913 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aefd402-83dc-40da-a85c-c76b29d14088 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:37.115104 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:37.189471 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1717df60-e144-4dfa-b483-d7a6c9c926af None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:37.203833 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aefd402-83dc-40da-a85c-c76b29d14088 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:37.229411 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0432fea-683b-4f17-b331-bf213b50e735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:37.230330 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0432fea-683b-4f17-b331-bf213b50e735 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:37.230690 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0432fea-683b-4f17-b331-bf213b50e735 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:37.231087 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0432fea-683b-4f17-b331-bf213b50e735 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:37.231932 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0432fea-683b-4f17-b331-bf213b50e735 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:37.233533 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0432fea-683b-4f17-b331-bf213b50e735 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:37.243869 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da398c44-0ffe-4ef5-a6a4-caa68e9573c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:37.244895 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da398c44-0ffe-4ef5-a6a4-caa68e9573c6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:37.245391 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da398c44-0ffe-4ef5-a6a4-caa68e9573c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:37.245878 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da398c44-0ffe-4ef5-a6a4-caa68e9573c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:37.246947 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-f0432fea-683b-4f17-b331-bf213b50e735 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 15:01:37.246947 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:37.246947 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 15:01:37.246947 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 15:01:37.246947 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 15:01:37.246947 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 15:01:37.246947 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 15:01:37.246947 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:01:37.246947 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 15:01:37.246947 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:37.246947 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 15:01:37.246947 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:37.246947 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:37.246947 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:37.246947 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:37.246947 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:37.247889 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:37.247889 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:37.247889 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:37.247889 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:37.247889 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:37.247889 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:37.247889 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:37.247889 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 15:01:37.247889 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 15:01:37.247889 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 15:01:37.247889 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 15:01:37.247889 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:37.247889 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:37.247889 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:37.247889 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:37.247889 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:37.248799 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:37.248799 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:37.248799 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:37.248799 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:37.248799 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:37.248799 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:37.248799 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:37.248799 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:37.248799 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:37.248799 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:37.248799 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:37.248799 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 15:01:37.248799 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:01:37.248799 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 15:01:37.248799 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:37.253278 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 389/764] 10.222.0.50 () {56 vars in 1129 bytes} [Fri May 27 15:01:37 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:37.254177 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0432fea-683b-4f17-b331-bf213b50e735 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:37.257168 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:37.257905 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da398c44-0ffe-4ef5-a6a4-caa68e9573c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:37.259625 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da398c44-0ffe-4ef5-a6a4-caa68e9573c6 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:37.275495 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-da398c44-0ffe-4ef5-a6a4-caa68e9573c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:37.275495 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:37.275495 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:37.275495 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:37.275495 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:37.275495 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:37.275495 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:37.275495 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:37.275495 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:37.275495 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:37.275495 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:37.275495 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:37.275495 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 15:01:37.275495 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 15:01:37.275495 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 15:01:37.276429 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 15:01:37.276429 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:37.276429 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:37.276429 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:37.276429 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:37.276429 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:37.276429 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:37.276429 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:37.276429 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:37.276429 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:37.276429 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:37.276429 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:37.276429 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:37.276429 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:37.276429 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:37.276429 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:37.276429 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:37.277432 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 15:01:37.277432 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 15:01:37.277432 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 15:01:37.277432 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 15:01:37.277432 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 15:01:37.277432 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 15:01:37.277432 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:37.277432 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:37.284756 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 376/765] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:01:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:37.285910 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da398c44-0ffe-4ef5-a6a4-caa68e9573c6 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:37.366380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0432fea-683b-4f17-b331-bf213b50e735 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:37.375688 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:37.391740 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da398c44-0ffe-4ef5-a6a4-caa68e9573c6 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:37.405849 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4239ec3-92ee-4b8c-9ab6-cf29d25458f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:37.406564 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4239ec3-92ee-4b8c-9ab6-cf29d25458f5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:37.406918 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4239ec3-92ee-4b8c-9ab6-cf29d25458f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:37.407199 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4239ec3-92ee-4b8c-9ab6-cf29d25458f5 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:37.407988 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4239ec3-92ee-4b8c-9ab6-cf29d25458f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:37.408613 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4239ec3-92ee-4b8c-9ab6-cf29d25458f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:37.409836 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4239ec3-92ee-4b8c-9ab6-cf29d25458f5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:37.428049 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 390/766] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 15:01:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:37.429215 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4239ec3-92ee-4b8c-9ab6-cf29d25458f5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:37.429878 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da27196-b01a-42ad-b835-32c387f0d2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:37.430773 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da27196-b01a-42ad-b835-32c387f0d2a1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:37.431340 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da27196-b01a-42ad-b835-32c387f0d2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:37.431831 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da27196-b01a-42ad-b835-32c387f0d2a1 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:37.432739 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da27196-b01a-42ad-b835-32c387f0d2a1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:37.433253 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da27196-b01a-42ad-b835-32c387f0d2a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:37.434820 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da27196-b01a-42ad-b835-32c387f0d2a1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:37.440789 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 377/767] 10.222.0.50 () {56 vars in 1124 bytes} [Fri May 27 15:01:37 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:37.441858 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da27196-b01a-42ad-b835-32c387f0d2a1 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:37.504846 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:37.548494 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da27196-b01a-42ad-b835-32c387f0d2a1 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:37.548855 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4239ec3-92ee-4b8c-9ab6-cf29d25458f5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:37.585112 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e16dc4-e3c3-4890-84ae-d19671e82ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:37.586604 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e16dc4-e3c3-4890-84ae-d19671e82ef6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:37.586878 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d03bb48-06ab-4567-8bf2-a404bedfc1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:37.587348 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e16dc4-e3c3-4890-84ae-d19671e82ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:37.587538 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03bb48-06ab-4567-8bf2-a404bedfc1f4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:37.587892 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03bb48-06ab-4567-8bf2-a404bedfc1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:37.588430 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03bb48-06ab-4567-8bf2-a404bedfc1f4 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:37.588812 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e16dc4-e3c3-4890-84ae-d19671e82ef6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:37.589215 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d03bb48-06ab-4567-8bf2-a404bedfc1f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:37.589521 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d03bb48-06ab-4567-8bf2-a404bedfc1f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:37.590271 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e16dc4-e3c3-4890-84ae-d19671e82ef6 None admin] RBAC: Adding query filter params (role.id=13f37584013544e7ad1e2c8f350da6d2, scope.project.id=daf12585428b41a2b45bdbc85413a6c4, user.id=09c23d1c732c452a9597c647ad0241f0) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:37.590787 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e16dc4-e3c3-4890-84ae-d19671e82ef6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:37.590993 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d03bb48-06ab-4567-8bf2-a404bedfc1f4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:37.592637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e16dc4-e3c3-4890-84ae-d19671e82ef6 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:37.598935 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 378/768] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:01:37 2022] GET /identity/v3/role_assignments?user.id=09c23d1c732c452a9597c647ad0241f0&scope.project.id=daf12585428b41a2b45bdbc85413a6c4&role.id=13f37584013544e7ad1e2c8f350da6d2 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:37.599993 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e16dc4-e3c3-4890-84ae-d19671e82ef6 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:37.616318 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 391/769] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 15:01:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:37.617713 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d03bb48-06ab-4567-8bf2-a404bedfc1f4 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:37.623377 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:37.721116 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e16dc4-e3c3-4890-84ae-d19671e82ef6 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:37.757287 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d03bb48-06ab-4567-8bf2-a404bedfc1f4 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:37.759712 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3538d51c-859c-4184-a777-2356496f706d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:37.760645 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3538d51c-859c-4184-a777-2356496f706d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:37.761144 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3538d51c-859c-4184-a777-2356496f706d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:37.761635 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3538d51c-859c-4184-a777-2356496f706d None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:37.762610 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3538d51c-859c-4184-a777-2356496f706d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:37.763163 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3538d51c-859c-4184-a777-2356496f706d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:37.764843 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3538d51c-859c-4184-a777-2356496f706d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:37.770729 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 379/770] 10.222.0.50 () {56 vars in 1124 bytes} [Fri May 27 15:01:37 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:37.772581 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e0264-832a-43a2-832b-5852cf8de59d None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:37.773112 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e0264-832a-43a2-832b-5852cf8de59d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:37.773598 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e0264-832a-43a2-832b-5852cf8de59d None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:37.775346 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 380/771] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:37.776254 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5e0264-832a-43a2-832b-5852cf8de59d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:37.788166 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:37.803811 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0569b3-e27c-4ccf-85de-fdb65161bbcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:37.804732 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0569b3-e27c-4ccf-85de-fdb65161bbcf None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:37.805015 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0569b3-e27c-4ccf-85de-fdb65161bbcf None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:37.805292 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0569b3-e27c-4ccf-85de-fdb65161bbcf None admin] PATH_INFO: `/v3/projects/daf12585428b41a2b45bdbc85413a6c4/users/451a9232f95e494683d81b6078a318d0/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:37.807620 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0569b3-e27c-4ccf-85de-fdb65161bbcf None admin] RBAC: Authorizing `identity:create_grant(project_id=daf12585428b41a2b45bdbc85413a6c4, user_id=451a9232f95e494683d81b6078a318d0, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:37.809139 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0569b3-e27c-4ccf-85de-fdb65161bbcf None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:37.829696 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 392/772] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 15:01:37 2022] PUT /identity/v3/projects/daf12585428b41a2b45bdbc85413a6c4/users/451a9232f95e494683d81b6078a318d0/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:01:37.832425 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e8ca2-ab42-4f13-b2be-6ea5799c01f7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:37.832726 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e8ca2-ab42-4f13-b2be-6ea5799c01f7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:37.833091 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e8ca2-ab42-4f13-b2be-6ea5799c01f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:37.853962 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1d4e8ca2-ab42-4f13-b2be-6ea5799c01f7 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:37.855526 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1d4e8ca2-ab42-4f13-b2be-6ea5799c01f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:37.927961 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:37.972708 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5e0264-832a-43a2-832b-5852cf8de59d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:37.980586 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 393/773] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:01:38.008110 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e8ca2-ab42-4f13-b2be-6ea5799c01f7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:38.051515 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ccb14-c59f-4ad1-b690-1b77c5cb7e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:38.052198 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ccb14-c59f-4ad1-b690-1b77c5cb7e2a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:38.052465 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ccb14-c59f-4ad1-b690-1b77c5cb7e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:38.052742 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ccb14-c59f-4ad1-b690-1b77c5cb7e2a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:38.053557 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44ccb14-c59f-4ad1-b690-1b77c5cb7e2a None admin] RBAC: Adding query filter params (interface=public, service_id=2a897f5f3b0b4ab5b2614cd9d19d54f8, region_id=RegionOne) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:38.053845 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44ccb14-c59f-4ad1-b690-1b77c5cb7e2a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:38.055245 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44ccb14-c59f-4ad1-b690-1b77c5cb7e2a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:38.064636 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 381/774] 10.222.0.50 () {56 vars in 1251 bytes} [Fri May 27 15:01:37 2022] GET /identity/v3/endpoints?service_id=2a897f5f3b0b4ab5b2614cd9d19d54f8&interface=public®ion_id=RegionOne => generated 194 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:38.070553 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ccb14-c59f-4ad1-b690-1b77c5cb7e2a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:38.138787 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e8ca2-ab42-4f13-b2be-6ea5799c01f7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:38.161444 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:38.191853 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ccb14-c59f-4ad1-b690-1b77c5cb7e2a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:38.199754 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a27aaa0-769b-4275-8a1b-74378071201b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:38.200847 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27aaa0-769b-4275-8a1b-74378071201b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:38.201383 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27aaa0-769b-4275-8a1b-74378071201b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:38.201914 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27aaa0-769b-4275-8a1b-74378071201b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:38.232213 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a27aaa0-769b-4275-8a1b-74378071201b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:01:38.245546 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-4a27aaa0-769b-4275-8a1b-74378071201b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:01:38.245546 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:38.245546 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:38.245546 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:38.245546 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:38.245546 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:38.245546 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:38.245546 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:38.245546 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:38.245546 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:38.245546 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:38.245546 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:38.245546 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:01:38.245546 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:01:38.245546 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:01:38.245546 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:01:38.246571 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:01:38.246571 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:01:38.246571 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:01:38.246571 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:01:38.246571 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:38.246571 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:38.246571 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:38.246571 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:38.246571 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:38.246571 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:38.246571 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:38.246571 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:38.246571 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:38.246571 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:38.246571 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:38.246571 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:38.247448 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:38.247448 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:38.247448 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:38.247448 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:38.247448 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:01:38.247448 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:01:38.247448 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:01:38.247448 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:01:38.247448 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:01:38.247448 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:01:38.247448 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:01:38.247448 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:38.247448 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 394/775] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 15:01:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:01:38.255941 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a27aaa0-769b-4275-8a1b-74378071201b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:38.275850 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae13c729-bed6-432c-8724-38a899b0fb33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:38.276553 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13c729-bed6-432c-8724-38a899b0fb33 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:38.276840 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13c729-bed6-432c-8724-38a899b0fb33 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:38.277119 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae13c729-bed6-432c-8724-38a899b0fb33 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:38.277830 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae13c729-bed6-432c-8724-38a899b0fb33 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:38.279616 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae13c729-bed6-432c-8724-38a899b0fb33 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:38.290969 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 382/776] 10.222.0.50 () {56 vars in 1087 bytes} [Fri May 27 15:01:38 2022] GET /identity/v3/services => generated 772 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:01:38.433702 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a27aaa0-769b-4275-8a1b-74378071201b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:38.481554 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977852a4-6d3d-4fc3-ac2b-54bc5a908ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:38.482158 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977852a4-6d3d-4fc3-ac2b-54bc5a908ada None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:38.482439 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977852a4-6d3d-4fc3-ac2b-54bc5a908ada None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:38.482829 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977852a4-6d3d-4fc3-ac2b-54bc5a908ada None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:38.484051 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977852a4-6d3d-4fc3-ac2b-54bc5a908ada None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:38.484363 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977852a4-6d3d-4fc3-ac2b-54bc5a908ada None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:38.485737 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977852a4-6d3d-4fc3-ac2b-54bc5a908ada None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:38.497162 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 395/777] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:01:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:38.503190 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae13c729-bed6-432c-8724-38a899b0fb33 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:38.516782 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:38.583400 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39824f4a-9094-4fae-bf31-1f8c3657c0a2 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:38.583743 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39824f4a-9094-4fae-bf31-1f8c3657c0a2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:38.583958 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39824f4a-9094-4fae-bf31-1f8c3657c0a2 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:38.585680 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 396/778] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 15:01:38.591291 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d22178-a2cc-4baa-a379-c0678d3c1493 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:38.591578 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d22178-a2cc-4baa-a379-c0678d3c1493 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:38.591856 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d22178-a2cc-4baa-a379-c0678d3c1493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:38.612188 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-99d22178-a2cc-4baa-a379-c0678d3c1493 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:38.613376 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-99d22178-a2cc-4baa-a379-c0678d3c1493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:38.624999 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae13c729-bed6-432c-8724-38a899b0fb33 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:38.644120 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:38.665953 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b9f47-cd59-430c-b3fd-432be8299002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:38.666756 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b9f47-cd59-430c-b3fd-432be8299002 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:38.667069 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b9f47-cd59-430c-b3fd-432be8299002 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:38.667348 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b9f47-cd59-430c-b3fd-432be8299002 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:38.690787 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0b9f47-cd59-430c-b3fd-432be8299002 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:38.692273 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0b9f47-cd59-430c-b3fd-432be8299002 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:38.710203 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 397/779] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 15:01:38.724918 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-ea0b9f47-cd59-430c-b3fd-432be8299002 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:38.724918 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:38.724918 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:38.724918 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:38.724918 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:38.724918 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:38.724918 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:38.724918 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:38.724918 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:38.724918 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:38.724918 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:38.724918 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:38.724918 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:38.724918 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:38.724918 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:38.725830 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:38.725830 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:38.725830 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:38.725830 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:38.725830 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:38.725830 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:38.725830 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:38.725830 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:38.725830 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:38.725830 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:38.725830 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:38.725830 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:38.725830 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:38.725830 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:38.725830 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:38.725830 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:38.725830 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:38.726724 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:38.726724 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:38.726724 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:38.726724 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:38.726724 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 383/780] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:38.732031 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d22178-a2cc-4baa-a379-c0678d3c1493 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:38.755900 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b9f47-cd59-430c-b3fd-432be8299002 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:38.819350 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:38.881818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0b9f47-cd59-430c-b3fd-432be8299002 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:38.921931 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d22178-a2cc-4baa-a379-c0678d3c1493 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:38.953258 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b76fe1-774f-4c2f-80db-2f9d5df0ebf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:38.954069 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b76fe1-774f-4c2f-80db-2f9d5df0ebf8 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:38.954353 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b76fe1-774f-4c2f-80db-2f9d5df0ebf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:38.954630 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b76fe1-774f-4c2f-80db-2f9d5df0ebf8 None admin] PATH_INFO: `/v3/services/network` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:38.955329 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b76fe1-774f-4c2f-80db-2f9d5df0ebf8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:38.956714 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b76fe1-774f-4c2f-80db-2f9d5df0ebf8 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:38.979249 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a0df0-496b-451f-8073-da0776637808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:38.980223 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a0df0-496b-451f-8073-da0776637808 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:38.980712 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a0df0-496b-451f-8073-da0776637808 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:38.981189 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a0df0-496b-451f-8073-da0776637808 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:38.982181 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11a0df0-496b-451f-8073-da0776637808 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:38.982699 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11a0df0-496b-451f-8073-da0776637808 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:38.984316 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11a0df0-496b-451f-8073-da0776637808 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:38.985752 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-17b76fe1-774f-4c2f-80db-2f9d5df0ebf8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 27 15:01:38.985752 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:38.985752 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 15:01:38.985752 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 15:01:38.985752 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 15:01:38.985752 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 15:01:38.985752 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 15:01:38.985752 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:01:38.985752 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 15:01:38.985752 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:38.985752 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 15:01:38.985752 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:38.985752 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:38.985752 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:38.985752 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:38.985752 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:38.985752 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:38.986749 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:38.986749 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:38.986749 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:38.986749 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:38.986749 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:38.986749 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:38.986749 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 15:01:38.986749 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 15:01:38.986749 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 15:01:38.986749 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 15:01:38.986749 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:38.986749 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:38.986749 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:38.986749 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:38.986749 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:38.986749 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:38.987612 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:38.987612 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:38.987612 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:38.987612 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:38.987612 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:38.987612 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:38.987612 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:38.987612 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:38.987612 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:38.987612 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:38.987612 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 15:01:38.987612 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:01:38.987612 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 15:01:38.987612 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:38.996282 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 384/781] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:01:38 2022] GET /identity/v3/services/network => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:39.004395 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b76fe1-774f-4c2f-80db-2f9d5df0ebf8 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:39.022923 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 398/782] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:39.028446 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a0df0-496b-451f-8073-da0776637808 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:39.083224 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:39.206099 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b76fe1-774f-4c2f-80db-2f9d5df0ebf8 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:39.224736 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a0df0-496b-451f-8073-da0776637808 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:39.233638 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:39.248350 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d8fb5c-cc8e-44c2-8659-39d56a1ee53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:39.250227 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d8fb5c-cc8e-44c2-8659-39d56a1ee53f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:39.250750 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d8fb5c-cc8e-44c2-8659-39d56a1ee53f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:39.251288 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d8fb5c-cc8e-44c2-8659-39d56a1ee53f None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:39.252207 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d8fb5c-cc8e-44c2-8659-39d56a1ee53f None admin] RBAC: Adding query filter params (name=network) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:39.252714 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d8fb5c-cc8e-44c2-8659-39d56a1ee53f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:39.254297 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d8fb5c-cc8e-44c2-8659-39d56a1ee53f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:39.259697 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 385/783] 10.222.0.50 () {56 vars in 1112 bytes} [Fri May 27 15:01:39 2022] GET /identity/v3/services?name=network => generated 124 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:01:39.265585 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d8fb5c-cc8e-44c2-8659-39d56a1ee53f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:39.304498 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055141d6-4021-411e-b46d-063ddc4979a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:39.305320 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055141d6-4021-411e-b46d-063ddc4979a7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:39.305596 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055141d6-4021-411e-b46d-063ddc4979a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:39.305872 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055141d6-4021-411e-b46d-063ddc4979a7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:39.324061 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055141d6-4021-411e-b46d-063ddc4979a7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:39.325569 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055141d6-4021-411e-b46d-063ddc4979a7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:39.340207 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-055141d6-4021-411e-b46d-063ddc4979a7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 15:01:39.340207 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:39.340207 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:39.340207 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:39.340207 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:39.340207 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:39.340207 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:39.340207 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:39.340207 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:39.340207 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:39.340207 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:39.340207 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:39.340207 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 15:01:39.340207 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 15:01:39.340207 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 15:01:39.340207 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 15:01:39.341179 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:39.341179 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:39.341179 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 15:01:39.341179 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:39.341179 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 15:01:39.341179 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:39.341179 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:39.341179 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:39.341179 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:39.341179 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:39.341179 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:39.341179 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:39.341179 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:39.341179 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:39.341179 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:39.341179 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:39.341179 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:39.341179 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:39.342137 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:39.342137 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:39.342137 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 15:01:39.342137 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 15:01:39.342137 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 15:01:39.342137 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 15:01:39.342137 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 15:01:39.342137 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 15:01:39.342137 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 15:01:39.342137 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:39.342137 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 399/784] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 15:01:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:01:39.347286 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055141d6-4021-411e-b46d-063ddc4979a7 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:39.413287 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d8fb5c-cc8e-44c2-8659-39d56a1ee53f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:39.468689 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914b27df-7799-4915-9b5a-c27b2e69f08e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:39.469594 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b27df-7799-4915-9b5a-c27b2e69f08e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:39.469891 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b27df-7799-4915-9b5a-c27b2e69f08e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:39.470177 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b27df-7799-4915-9b5a-c27b2e69f08e None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:39.470912 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914b27df-7799-4915-9b5a-c27b2e69f08e None admin] RBAC: Adding query filter params (type=network) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:39.471228 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914b27df-7799-4915-9b5a-c27b2e69f08e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:39.472800 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914b27df-7799-4915-9b5a-c27b2e69f08e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:39.478266 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 386/785] 10.222.0.50 () {56 vars in 1112 bytes} [Fri May 27 15:01:39 2022] GET /identity/v3/services?type=network => generated 350 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:39.484415 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914b27df-7799-4915-9b5a-c27b2e69f08e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:39.502169 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:39.519940 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055141d6-4021-411e-b46d-063ddc4979a7 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:39.567262 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:39.571542 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89a827-7d6d-4d4b-a7bd-e2b52ac81476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:39.572228 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89a827-7d6d-4d4b-a7bd-e2b52ac81476 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:39.572515 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89a827-7d6d-4d4b-a7bd-e2b52ac81476 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:39.572794 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89a827-7d6d-4d4b-a7bd-e2b52ac81476 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:39.573641 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b89a827-7d6d-4d4b-a7bd-e2b52ac81476 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:39.574076 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b89a827-7d6d-4d4b-a7bd-e2b52ac81476 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:39.575387 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b89a827-7d6d-4d4b-a7bd-e2b52ac81476 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:39.621129 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914b27df-7799-4915-9b5a-c27b2e69f08e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:39.625397 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 400/786] 10.222.0.50 () {56 vars in 1135 bytes} [Fri May 27 15:01:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:01:39.632631 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89a827-7d6d-4d4b-a7bd-e2b52ac81476 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:39.665470 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74114011-2e88-4e51-82a9-cfc7a2322688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:39.666794 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74114011-2e88-4e51-82a9-cfc7a2322688 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:39.667293 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74114011-2e88-4e51-82a9-cfc7a2322688 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:39.667748 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74114011-2e88-4e51-82a9-cfc7a2322688 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:39.668742 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74114011-2e88-4e51-82a9-cfc7a2322688 None admin] RBAC: Adding query filter params (interface=public, service_id=da71440ded4f4e63b0b844e4017a5598, region_id=RegionOne) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:39.669246 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74114011-2e88-4e51-82a9-cfc7a2322688 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:39.671509 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74114011-2e88-4e51-82a9-cfc7a2322688 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:39.678019 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 387/787] 10.222.0.50 () {56 vars in 1251 bytes} [Fri May 27 15:01:39 2022] GET /identity/v3/endpoints?service_id=da71440ded4f4e63b0b844e4017a5598&interface=public®ion_id=RegionOne => generated 194 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:39.683203 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74114011-2e88-4e51-82a9-cfc7a2322688 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:39.797134 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74114011-2e88-4e51-82a9-cfc7a2322688 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:39.812138 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:39.844945 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89a827-7d6d-4d4b-a7bd-e2b52ac81476 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:39.855091 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1713b09d-4814-4dcd-be6d-0c88272491b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:39.855864 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713b09d-4814-4dcd-be6d-0c88272491b7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:39.856155 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713b09d-4814-4dcd-be6d-0c88272491b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:39.856432 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713b09d-4814-4dcd-be6d-0c88272491b7 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:39.857122 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1713b09d-4814-4dcd-be6d-0c88272491b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:39.858768 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1713b09d-4814-4dcd-be6d-0c88272491b7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:39.864755 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 388/788] 10.222.0.50 () {56 vars in 1087 bytes} [Fri May 27 15:01:39 2022] GET /identity/v3/services => generated 772 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:39.889046 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5b7a28-5ace-4734-bba8-04e85c1f230b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:39.890417 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b7a28-5ace-4734-bba8-04e85c1f230b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:39.890693 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b7a28-5ace-4734-bba8-04e85c1f230b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:39.890987 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5b7a28-5ace-4734-bba8-04e85c1f230b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:39.905910 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5b7a28-5ace-4734-bba8-04e85c1f230b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:39.915718 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5b7a28-5ace-4734-bba8-04e85c1f230b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:39.917881 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:39.943446 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-7a5b7a28-5ace-4734-bba8-04e85c1f230b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:39.943446 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:39.943446 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:39.943446 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:39.943446 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:39.943446 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:39.943446 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:39.943446 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:39.943446 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:39.943446 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:39.943446 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:39.943446 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:39.943446 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:39.943446 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:39.943446 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:39.944508 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:39.944508 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:39.944508 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:39.944508 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:39.944508 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:39.944508 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:39.944508 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:39.944508 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:39.944508 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:39.944508 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:39.944508 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:39.944508 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:39.944508 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:39.944508 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:39.944508 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:39.944508 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:39.944508 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:39.945420 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:39.945420 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:39.945420 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:39.945420 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:39.949320 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 401/789] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:39.950760 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1713b09d-4814-4dcd-be6d-0c88272491b7 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:40.129244 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1713b09d-4814-4dcd-be6d-0c88272491b7 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:40.165560 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b954a482-a456-476c-b8ac-fe09464b1a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:40.166370 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954a482-a456-476c-b8ac-fe09464b1a56 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:40.166756 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954a482-a456-476c-b8ac-fe09464b1a56 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:40.167157 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b954a482-a456-476c-b8ac-fe09464b1a56 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:40.167990 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b954a482-a456-476c-b8ac-fe09464b1a56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:40.168396 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b954a482-a456-476c-b8ac-fe09464b1a56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:40.169861 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b954a482-a456-476c-b8ac-fe09464b1a56 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:40.183315 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 389/790] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:40.189241 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5b7a28-5ace-4734-bba8-04e85c1f230b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:40.306439 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5b7a28-5ace-4734-bba8-04e85c1f230b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:40.351642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d13b09-065d-4f63-a24a-04fd0e854059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:40.352424 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d13b09-065d-4f63-a24a-04fd0e854059 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:40.352704 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d13b09-065d-4f63-a24a-04fd0e854059 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:40.352972 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d13b09-065d-4f63-a24a-04fd0e854059 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:40.360780 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d13b09-065d-4f63-a24a-04fd0e854059 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:40.362222 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d13b09-065d-4f63-a24a-04fd0e854059 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:40.381623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-45d13b09-065d-4f63-a24a-04fd0e854059 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:40.381623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:40.381623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:40.381623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:40.381623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:40.381623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:40.381623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:40.381623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:40.381623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:40.381623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:40.381623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:40.381623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:40.381623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 15:01:40.381623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 15:01:40.381623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 15:01:40.382568 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 15:01:40.382568 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:40.382568 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:40.382568 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:40.382568 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:40.382568 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:40.382568 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:40.382568 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:40.382568 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:40.382568 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:40.382568 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:40.382568 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:40.382568 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:40.382568 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:40.382568 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:40.382568 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:40.382568 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:40.383517 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 15:01:40.383517 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 15:01:40.383517 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 15:01:40.383517 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 15:01:40.383517 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 15:01:40.383517 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 15:01:40.383517 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:40.383517 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:40.387177 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 402/791] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:01:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:40.388973 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b954a482-a456-476c-b8ac-fe09464b1a56 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:40.438650 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b696a47-fc69-4e0f-835e-4c172a7f83b6 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:40.438989 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b696a47-fc69-4e0f-835e-4c172a7f83b6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:40.439231 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b696a47-fc69-4e0f-835e-4c172a7f83b6 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:40.440596 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 403/792] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 15:01:40.457562 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce344271-9994-4747-b331-31d70ecc2e28 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:40.457878 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce344271-9994-4747-b331-31d70ecc2e28 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:40.458105 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce344271-9994-4747-b331-31d70ecc2e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:40.468954 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:40.485448 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-ce344271-9994-4747-b331-31d70ecc2e28 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:40.486929 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ce344271-9994-4747-b331-31d70ecc2e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:40.536402 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b954a482-a456-476c-b8ac-fe09464b1a56 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:40.538131 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:40.560243 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 404/793] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:01:40.561640 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffccd68-2e3a-4491-83d6-b36411e839ad None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:40.561938 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffccd68-2e3a-4491-83d6-b36411e839ad None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:40.562219 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffccd68-2e3a-4491-83d6-b36411e839ad None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:40.572175 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 405/794] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:40 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:40.578107 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f955fd-8d39-4578-b131-69656ba670a2 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:40.578407 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f955fd-8d39-4578-b131-69656ba670a2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:40.578697 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f955fd-8d39-4578-b131-69656ba670a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:40.580055 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fff0cf-0ab8-42e1-afaa-cb7158ea9c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:40.580981 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fff0cf-0ab8-42e1-afaa-cb7158ea9c48 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:40.581503 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fff0cf-0ab8-42e1-afaa-cb7158ea9c48 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:40.581983 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fff0cf-0ab8-42e1-afaa-cb7158ea9c48 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:40.583097 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fff0cf-0ab8-42e1-afaa-cb7158ea9c48 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:40.583678 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fff0cf-0ab8-42e1-afaa-cb7158ea9c48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:40.585394 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fff0cf-0ab8-42e1-afaa-cb7158ea9c48 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:40.599611 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 390/795] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 15:01:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:40.605276 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fff0cf-0ab8-42e1-afaa-cb7158ea9c48 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:40.606272 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-26f955fd-8d39-4578-b131-69656ba670a2 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:40.608743 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-26f955fd-8d39-4578-b131-69656ba670a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:40.714975 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 406/796] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:01:40.714975 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f955fd-8d39-4578-b131-69656ba670a2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:40.753604 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:40.758427 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fff0cf-0ab8-42e1-afaa-cb7158ea9c48 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:40.802903 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eb6252-19b1-413b-9aa4-b70e8bd356e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:40.804391 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eb6252-19b1-413b-9aa4-b70e8bd356e4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:40.805000 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eb6252-19b1-413b-9aa4-b70e8bd356e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:40.805352 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eb6252-19b1-413b-9aa4-b70e8bd356e4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:40.815846 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f955fd-8d39-4578-b131-69656ba670a2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:40.820793 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eb6252-19b1-413b-9aa4-b70e8bd356e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:01:40.831954 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:40.832949 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-82eb6252-19b1-413b-9aa4-b70e8bd356e4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 15:01:40.832949 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:40.832949 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:40.832949 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:40.832949 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:40.832949 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:40.832949 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:40.832949 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:40.832949 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:40.832949 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:40.832949 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:40.832949 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:40.832949 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:01:40.832949 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:01:40.832949 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:01:40.832949 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:01:40.833921 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:01:40.833921 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:01:40.833921 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:01:40.833921 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:01:40.833921 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:40.833921 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:40.833921 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:40.833921 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:40.833921 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:40.833921 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:40.833921 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:40.833921 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:40.833921 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:40.833921 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:40.833921 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:40.833921 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:40.834776 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:40.834776 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:40.834776 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:40.834776 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:40.834776 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:01:40.834776 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:01:40.834776 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:01:40.834776 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:01:40.834776 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:01:40.834776 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:01:40.834776 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 15:01:40.834776 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:40.839835 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 391/797] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 15:01:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:40.839835 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eb6252-19b1-413b-9aa4-b70e8bd356e4 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:40.862437 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87bb7b7-fda8-4ca0-975b-6c7bad383234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:40.863377 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87bb7b7-fda8-4ca0-975b-6c7bad383234 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:40.863662 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87bb7b7-fda8-4ca0-975b-6c7bad383234 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:40.863942 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87bb7b7-fda8-4ca0-975b-6c7bad383234 None admin] PATH_INFO: `/v3/projects/daf12585428b41a2b45bdbc85413a6c4/users/18903f2eab7c415096e075d47f0b9a4d/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:40.865973 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87bb7b7-fda8-4ca0-975b-6c7bad383234 None admin] RBAC: Authorizing `identity:create_grant(project_id=daf12585428b41a2b45bdbc85413a6c4, user_id=18903f2eab7c415096e075d47f0b9a4d, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:40.867456 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87bb7b7-fda8-4ca0-975b-6c7bad383234 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:40.880764 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 407/798] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 15:01:40 2022] PUT /identity/v3/projects/daf12585428b41a2b45bdbc85413a6c4/users/18903f2eab7c415096e075d47f0b9a4d/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:01:40.882958 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87bb7b7-fda8-4ca0-975b-6c7bad383234 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:40.984693 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eb6252-19b1-413b-9aa4-b70e8bd356e4 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:41.028715 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c16509-a431-4a3d-9b69-ebe538869b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:41.029541 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c16509-a431-4a3d-9b69-ebe538869b77 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:41.030361 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c16509-a431-4a3d-9b69-ebe538869b77 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:41.030715 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c16509-a431-4a3d-9b69-ebe538869b77 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:41.031654 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87bb7b7-fda8-4ca0-975b-6c7bad383234 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:41.043548 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c16509-a431-4a3d-9b69-ebe538869b77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:01:41.051699 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-33c16509-a431-4a3d-9b69-ebe538869b77 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:01:41.051699 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:41.051699 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:41.051699 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:41.051699 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:41.051699 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:41.051699 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:41.051699 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:41.051699 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:41.051699 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:41.051699 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:41.051699 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:41.051699 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:01:41.051699 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:01:41.051699 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:01:41.051699 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:01:41.052709 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:01:41.052709 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:01:41.052709 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:01:41.052709 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:01:41.052709 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:41.052709 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:41.052709 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:41.052709 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:41.052709 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:41.052709 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:41.052709 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:41.052709 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:41.052709 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:41.052709 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:41.052709 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:41.052709 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:41.053558 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:41.053558 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:41.053558 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:41.053558 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:41.053558 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:01:41.053558 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:01:41.053558 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:01:41.053558 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:01:41.053558 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:01:41.053558 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:01:41.053558 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:01:41.053558 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:41.058063 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 392/799] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 15:01:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:41.059514 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffa8495-e9ee-4fe2-ae74-df73bcd30915 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:41.059806 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffa8495-e9ee-4fe2-ae74-df73bcd30915 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:41.060070 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffa8495-e9ee-4fe2-ae74-df73bcd30915 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:41.062490 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 393/800] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:41.063331 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffa8495-e9ee-4fe2-ae74-df73bcd30915 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:41.086303 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:41.091450 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e2d2dc-1311-4220-9883-579bdc0623a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:41.092137 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2d2dc-1311-4220-9883-579bdc0623a2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:41.092292 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2d2dc-1311-4220-9883-579bdc0623a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:41.093473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e2d2dc-1311-4220-9883-579bdc0623a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:41.093473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e2d2dc-1311-4220-9883-579bdc0623a2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:41.093655 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e2d2dc-1311-4220-9883-579bdc0623a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:41.096150 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e2d2dc-1311-4220-9883-579bdc0623a2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:41.110296 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 408/801] 10.222.0.50 () {56 vars in 1103 bytes} [Fri May 27 15:01:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:41.111454 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4129e-27a5-44d7-aa4b-79cee9b4a2af None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:41.111743 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4129e-27a5-44d7-aa4b-79cee9b4a2af None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:41.112026 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4129e-27a5-44d7-aa4b-79cee9b4a2af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:41.131249 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d6c4129e-27a5-44d7-aa4b-79cee9b4a2af None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:41.132479 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d6c4129e-27a5-44d7-aa4b-79cee9b4a2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:41.163044 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:41.209419 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffa8495-e9ee-4fe2-ae74-df73bcd30915 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:41.215405 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 409/802] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:01:41.222873 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c4129e-27a5-44d7-aa4b-79cee9b4a2af None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:41.269858 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff29607-1869-4140-8196-f9a781379150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:41.271337 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff29607-1869-4140-8196-f9a781379150 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:41.271770 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff29607-1869-4140-8196-f9a781379150 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:41.272193 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff29607-1869-4140-8196-f9a781379150 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:41.273562 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff29607-1869-4140-8196-f9a781379150 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:41.274423 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff29607-1869-4140-8196-f9a781379150 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:41.276268 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff29607-1869-4140-8196-f9a781379150 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:41.289614 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 394/803] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:01:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:41.290770 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff29607-1869-4140-8196-f9a781379150 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:41.342143 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c4129e-27a5-44d7-aa4b-79cee9b4a2af None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:41.389983 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f004e55d-30a0-4d85-b625-30c1b6fdc004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:41.391939 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004e55d-30a0-4d85-b625-30c1b6fdc004 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:41.392373 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004e55d-30a0-4d85-b625-30c1b6fdc004 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:41.392794 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f004e55d-30a0-4d85-b625-30c1b6fdc004 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:41.407226 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f004e55d-30a0-4d85-b625-30c1b6fdc004 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:41.409717 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f004e55d-30a0-4d85-b625-30c1b6fdc004 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:41.414968 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:41.435148 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-f004e55d-30a0-4d85-b625-30c1b6fdc004 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:41.435148 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:41.435148 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:41.435148 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:41.435148 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:41.435148 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:41.435148 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:41.435148 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:41.435148 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:41.435148 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:41.435148 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:41.435148 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:41.435148 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:41.435148 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:41.435148 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:41.436173 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:41.436173 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:41.436173 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:41.436173 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:41.436173 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:41.436173 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:41.436173 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:41.436173 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:41.436173 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:41.436173 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:41.436173 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:41.436173 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:41.436173 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:41.436173 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:41.436173 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:41.436173 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:41.436173 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:41.437404 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:41.437404 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:41.437404 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:41.437404 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:41.441275 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 410/804] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:41.442197 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f004e55d-30a0-4d85-b625-30c1b6fdc004 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:41.444330 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff29607-1869-4140-8196-f9a781379150 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:41.464757 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:41.493021 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f037e3ff-6e66-42aa-826a-067d2debd91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:41.494418 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037e3ff-6e66-42aa-826a-067d2debd91e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:41.494907 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037e3ff-6e66-42aa-826a-067d2debd91e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:41.495621 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f037e3ff-6e66-42aa-826a-067d2debd91e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:41.496937 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f037e3ff-6e66-42aa-826a-067d2debd91e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:41.499048 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f037e3ff-6e66-42aa-826a-067d2debd91e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:41.516678 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-f037e3ff-6e66-42aa-826a-067d2debd91e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 15:01:41.516678 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:41.516678 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 15:01:41.516678 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 15:01:41.516678 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 15:01:41.516678 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 15:01:41.516678 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 15:01:41.516678 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:01:41.516678 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 15:01:41.516678 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:41.516678 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 15:01:41.516678 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:41.516678 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:41.516678 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:41.516678 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:41.516678 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:41.517636 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:41.517636 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:41.517636 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:41.517636 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:41.517636 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:41.517636 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:41.517636 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:41.517636 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 15:01:41.517636 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 15:01:41.517636 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 15:01:41.517636 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 15:01:41.517636 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:41.517636 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:41.517636 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:41.517636 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:41.517636 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:41.518502 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:41.518502 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:41.518502 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:41.518502 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:41.518502 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:41.518502 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:41.518502 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:41.518502 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:41.518502 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:41.518502 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:41.518502 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:41.518502 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 15:01:41.518502 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:01:41.518502 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 27 15:01:41.518502 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:41.525114 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 395/805] 10.222.0.50 () {56 vars in 1129 bytes} [Fri May 27 15:01:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:41.525114 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f037e3ff-6e66-42aa-826a-067d2debd91e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:41.568505 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f004e55d-30a0-4d85-b625-30c1b6fdc004 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:41.607011 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c378a9-075c-4784-87bd-42c49a6945e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:41.608023 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c378a9-075c-4784-87bd-42c49a6945e4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:41.608318 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c378a9-075c-4784-87bd-42c49a6945e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:41.608687 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c378a9-075c-4784-87bd-42c49a6945e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:41.619367 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c378a9-075c-4784-87bd-42c49a6945e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:41.620973 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c378a9-075c-4784-87bd-42c49a6945e4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:41.639869 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-81c378a9-075c-4784-87bd-42c49a6945e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:41.639869 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:41.639869 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:41.639869 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:41.639869 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:41.639869 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:41.639869 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:41.639869 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:41.639869 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:41.639869 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:41.639869 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:41.639869 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:41.639869 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:41.639869 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:41.639869 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:41.640813 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:41.640813 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:41.640813 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:41.640813 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:41.640813 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:41.640813 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:41.640813 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:41.640813 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:41.640813 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:41.640813 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:41.640813 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:41.640813 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:41.640813 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:41.640813 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:41.640813 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:41.640813 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:41.640813 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:41.641702 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:41.641702 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:41.641702 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:41.641702 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:41.641702 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 411/806] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:41.642529 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c378a9-075c-4784-87bd-42c49a6945e4 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:41.676639 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f037e3ff-6e66-42aa-826a-067d2debd91e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:41.719551 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e058ec-4223-4ec9-beb4-7b85a18b1131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:41.720227 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e058ec-4223-4ec9-beb4-7b85a18b1131 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:41.720703 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e058ec-4223-4ec9-beb4-7b85a18b1131 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:41.721050 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e058ec-4223-4ec9-beb4-7b85a18b1131 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:41.721710 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e058ec-4223-4ec9-beb4-7b85a18b1131 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:41.722017 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e058ec-4223-4ec9-beb4-7b85a18b1131 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:41.723434 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e058ec-4223-4ec9-beb4-7b85a18b1131 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:41.738130 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 396/807] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:41.739517 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e058ec-4223-4ec9-beb4-7b85a18b1131 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:41.743421 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:41.748019 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:41.795656 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c378a9-075c-4784-87bd-42c49a6945e4 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:41.833116 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e7fb0-8690-420a-a628-6d6ce80ba1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:41.833781 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e7fb0-8690-420a-a628-6d6ce80ba1d5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:41.834063 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e7fb0-8690-420a-a628-6d6ce80ba1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:41.834336 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e7fb0-8690-420a-a628-6d6ce80ba1d5 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:41.835072 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10e7fb0-8690-420a-a628-6d6ce80ba1d5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:41.835382 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10e7fb0-8690-420a-a628-6d6ce80ba1d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:41.836741 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10e7fb0-8690-420a-a628-6d6ce80ba1d5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:41.846570 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 412/808] 10.222.0.50 () {56 vars in 1124 bytes} [Fri May 27 15:01:41 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:41.846570 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e7fb0-8690-420a-a628-6d6ce80ba1d5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:41.881469 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e058ec-4223-4ec9-beb4-7b85a18b1131 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:41.917683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b95b640-8e42-47eb-82ac-9d88764a944e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:41.918539 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b95b640-8e42-47eb-82ac-9d88764a944e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:41.919167 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b95b640-8e42-47eb-82ac-9d88764a944e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:41.919288 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b95b640-8e42-47eb-82ac-9d88764a944e None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:41.920123 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b95b640-8e42-47eb-82ac-9d88764a944e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:41.921482 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b95b640-8e42-47eb-82ac-9d88764a944e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:41.923100 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b95b640-8e42-47eb-82ac-9d88764a944e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:41.935221 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 397/809] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:41.936294 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b95b640-8e42-47eb-82ac-9d88764a944e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:41.980388 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e7fb0-8690-420a-a628-6d6ce80ba1d5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:42.013773 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:42.020733 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f6b6f1-3419-4a1b-a582-110a11486546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:42.021691 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6b6f1-3419-4a1b-a582-110a11486546 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:42.022078 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6b6f1-3419-4a1b-a582-110a11486546 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:42.022450 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f6b6f1-3419-4a1b-a582-110a11486546 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:42.024926 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:42.035944 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f6b6f1-3419-4a1b-a582-110a11486546 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:42.038219 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f6b6f1-3419-4a1b-a582-110a11486546 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:42.058130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-10f6b6f1-3419-4a1b-a582-110a11486546 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 15:01:42.058130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:42.058130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:42.058130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:42.058130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:42.058130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:42.058130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:42.058130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:42.058130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:42.058130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:42.058130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:42.058130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:42.058130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 15:01:42.058130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 15:01:42.058130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 15:01:42.058130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 15:01:42.059402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:42.059402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:42.059402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 15:01:42.059402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:42.059402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 15:01:42.059402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:42.059402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:42.059402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:42.059402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:42.059402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:42.059402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:42.059402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:42.059402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:42.059402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:42.059402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:42.059402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:42.059402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:42.059402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:42.060492 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:42.060492 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:42.060492 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 15:01:42.060492 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 15:01:42.060492 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 15:01:42.060492 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 15:01:42.060492 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 15:01:42.060492 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 15:01:42.060492 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 15:01:42.060492 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:42.063576 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 413/810] 10.222.0.50 () {56 vars in 1117 bytes} [Fri May 27 15:01:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:42.064637 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f6b6f1-3419-4a1b-a582-110a11486546 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:42.072759 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b95b640-8e42-47eb-82ac-9d88764a944e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:42.122891 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acbeca9-6eda-4cca-bb51-40c07c4aff90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:42.124164 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acbeca9-6eda-4cca-bb51-40c07c4aff90 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:42.125429 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acbeca9-6eda-4cca-bb51-40c07c4aff90 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:42.126094 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acbeca9-6eda-4cca-bb51-40c07c4aff90 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:42.127824 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acbeca9-6eda-4cca-bb51-40c07c4aff90 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:42.128331 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acbeca9-6eda-4cca-bb51-40c07c4aff90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:42.130814 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acbeca9-6eda-4cca-bb51-40c07c4aff90 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:42.143192 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 398/811] 10.222.0.50 () {56 vars in 1124 bytes} [Fri May 27 15:01:41 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:42.143192 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acbeca9-6eda-4cca-bb51-40c07c4aff90 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:42.203573 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f6b6f1-3419-4a1b-a582-110a11486546 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:42.246327 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4338b3-e3b1-410d-8d50-04b9898dd90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:42.247767 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4338b3-e3b1-410d-8d50-04b9898dd90a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:42.248598 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4338b3-e3b1-410d-8d50-04b9898dd90a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:42.249986 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4338b3-e3b1-410d-8d50-04b9898dd90a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:42.259840 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4338b3-e3b1-410d-8d50-04b9898dd90a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:42.262615 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4338b3-e3b1-410d-8d50-04b9898dd90a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:42.283528 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acbeca9-6eda-4cca-bb51-40c07c4aff90 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:42.285397 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-ef4338b3-e3b1-410d-8d50-04b9898dd90a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 15:01:42.285397 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:42.285397 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:42.285397 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:42.285397 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:42.285397 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:42.285397 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:42.285397 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:42.285397 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:42.285397 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:42.285397 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:42.285397 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:42.285397 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 15:01:42.285397 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 15:01:42.285397 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 15:01:42.285397 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 15:01:42.286330 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:42.286330 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:42.286330 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 15:01:42.286330 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:42.286330 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 15:01:42.286330 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:42.286330 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:42.286330 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:42.286330 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:42.286330 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:42.286330 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:42.286330 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:42.286330 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:42.286330 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:42.286330 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:42.286330 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:42.286330 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:42.286330 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:42.287361 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:42.287361 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:42.287361 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 15:01:42.287361 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 15:01:42.287361 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 15:01:42.287361 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 15:01:42.287361 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 15:01:42.287361 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 15:01:42.287361 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 15:01:42.287361 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:42.294964 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 414/812] 10.222.0.50 () {56 vars in 1093 bytes} [Fri May 27 15:01:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:42.296020 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4338b3-e3b1-410d-8d50-04b9898dd90a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:42.324656 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:42.342013 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e6fcfa-916d-4904-b9b3-2a0f8a2283b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:42.343859 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6fcfa-916d-4904-b9b3-2a0f8a2283b3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:42.344150 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6fcfa-916d-4904-b9b3-2a0f8a2283b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:42.344429 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6fcfa-916d-4904-b9b3-2a0f8a2283b3 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:42.345276 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e6fcfa-916d-4904-b9b3-2a0f8a2283b3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:42.345588 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e6fcfa-916d-4904-b9b3-2a0f8a2283b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:42.347067 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e6fcfa-916d-4904-b9b3-2a0f8a2283b3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:42.362447 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:42.369685 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 399/813] 10.222.0.50 () {56 vars in 1149 bytes} [Fri May 27 15:01:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:42.370798 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e6fcfa-916d-4904-b9b3-2a0f8a2283b3 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:42.416130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4338b3-e3b1-410d-8d50-04b9898dd90a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:42.459696 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760c5cd9-d2e5-4164-99bf-ad053e38a424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:42.461056 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760c5cd9-d2e5-4164-99bf-ad053e38a424 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:42.461580 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760c5cd9-d2e5-4164-99bf-ad053e38a424 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:42.462041 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760c5cd9-d2e5-4164-99bf-ad053e38a424 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:42.463565 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760c5cd9-d2e5-4164-99bf-ad053e38a424 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:42.465742 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760c5cd9-d2e5-4164-99bf-ad053e38a424 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:42.487486 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e6fcfa-916d-4904-b9b3-2a0f8a2283b3 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:42.490780 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 415/814] 10.222.0.50 () {60 vars in 1144 bytes} [Fri May 27 15:01:42 2022] POST /identity/v3/endpoints => generated 346 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:01:42.499985 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760c5cd9-d2e5-4164-99bf-ad053e38a424 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:42.543318 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e506fec-2f8b-49c9-a269-bd92d7419ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:42.544664 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e506fec-2f8b-49c9-a269-bd92d7419ef9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:42.546168 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e506fec-2f8b-49c9-a269-bd92d7419ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:42.546655 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e506fec-2f8b-49c9-a269-bd92d7419ef9 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:42.547762 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e506fec-2f8b-49c9-a269-bd92d7419ef9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:42.548292 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e506fec-2f8b-49c9-a269-bd92d7419ef9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:42.549966 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e506fec-2f8b-49c9-a269-bd92d7419ef9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:42.579172 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 400/815] 10.222.0.50 () {56 vars in 1133 bytes} [Fri May 27 15:01:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:42.581088 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bedd7d3-55e1-4164-a5a6-8b99ba2acd81 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:42.581558 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bedd7d3-55e1-4164-a5a6-8b99ba2acd81 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:42.581999 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bedd7d3-55e1-4164-a5a6-8b99ba2acd81 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:42.587405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 401/816] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:42.589930 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bedd7d3-55e1-4164-a5a6-8b99ba2acd81 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:42.631061 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760c5cd9-d2e5-4164-99bf-ad053e38a424 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:42.644392 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:42.657937 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:42.676327 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c81dc-d2ad-4b4f-9ea8-cae1ce852e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:42.677278 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c81dc-d2ad-4b4f-9ea8-cae1ce852e71 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:42.677747 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c81dc-d2ad-4b4f-9ea8-cae1ce852e71 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:42.678052 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c81dc-d2ad-4b4f-9ea8-cae1ce852e71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:42.693467 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4c81dc-d2ad-4b4f-9ea8-cae1ce852e71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:42.694974 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4c81dc-d2ad-4b4f-9ea8-cae1ce852e71 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:42.717073 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-3a4c81dc-d2ad-4b4f-9ea8-cae1ce852e71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:42.717073 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:42.717073 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:42.717073 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:42.717073 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:42.717073 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:42.717073 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:42.717073 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:42.717073 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:42.717073 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:42.717073 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:42.717073 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:42.717073 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:42.717073 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:42.717073 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:42.718508 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:42.718508 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:42.718508 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:42.718508 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:42.718508 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:42.718508 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:42.718508 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:42.718508 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:42.718508 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:42.718508 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:42.718508 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:42.718508 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:42.718508 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:42.718508 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:42.718508 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:42.718508 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:42.718508 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:42.719502 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:42.719502 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:42.719502 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:42.719502 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:42.719502 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 416/817] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:42.719792 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc3bdb9-1a2b-43c3-9bbc-3ec5747a2bca None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:42.720661 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc3bdb9-1a2b-43c3-9bbc-3ec5747a2bca None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:42.720661 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc3bdb9-1a2b-43c3-9bbc-3ec5747a2bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:42.738688 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7fc3bdb9-1a2b-43c3-9bbc-3ec5747a2bca None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:42.739905 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7fc3bdb9-1a2b-43c3-9bbc-3ec5747a2bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:42.743363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bedd7d3-55e1-4164-a5a6-8b99ba2acd81 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:42.787075 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5f4dbc-5b94-4666-acbf-288a055ce673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:42.787976 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f4dbc-5b94-4666-acbf-288a055ce673 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:42.788410 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f4dbc-5b94-4666-acbf-288a055ce673 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:42.788831 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f4dbc-5b94-4666-acbf-288a055ce673 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:42.822984 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 417/818] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:01:42.824135 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc3bdb9-1a2b-43c3-9bbc-3ec5747a2bca None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:42.873261 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5f4dbc-5b94-4666-acbf-288a055ce673 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:42.875311 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5f4dbc-5b94-4666-acbf-288a055ce673 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:42.917297 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:42.931563 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:42.942770 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc3bdb9-1a2b-43c3-9bbc-3ec5747a2bca None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:42.979273 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411facec-ae81-48ad-a691-b45132674065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:42.980186 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411facec-ae81-48ad-a691-b45132674065 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:42.980618 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411facec-ae81-48ad-a691-b45132674065 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:42.981062 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411facec-ae81-48ad-a691-b45132674065 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:42.981952 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411facec-ae81-48ad-a691-b45132674065 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:42.982406 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411facec-ae81-48ad-a691-b45132674065 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:42.984571 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411facec-ae81-48ad-a691-b45132674065 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:42.984746 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 402/819] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:01:42 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 15:01:42.985649 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5f4dbc-5b94-4666-acbf-288a055ce673 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:42.997276 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 418/820] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:42.998390 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411facec-ae81-48ad-a691-b45132674065 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:43.092656 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5f4dbc-5b94-4666-acbf-288a055ce673 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:43.109764 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411facec-ae81-48ad-a691-b45132674065 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:43.129828 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4377562d-709f-42d9-b84d-7f90c1e9e6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:43.130554 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4377562d-709f-42d9-b84d-7f90c1e9e6c6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:43.130841 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4377562d-709f-42d9-b84d-7f90c1e9e6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:43.131165 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4377562d-709f-42d9-b84d-7f90c1e9e6c6 None admin] PATH_INFO: `/v3/services/network` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:43.131820 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4377562d-709f-42d9-b84d-7f90c1e9e6c6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:43.133197 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4377562d-709f-42d9-b84d-7f90c1e9e6c6 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:43.145382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-4377562d-709f-42d9-b84d-7f90c1e9e6c6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 27 15:01:43.145382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:43.145382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 15:01:43.145382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 15:01:43.145382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 15:01:43.145382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 15:01:43.145382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 15:01:43.145382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:01:43.145382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 15:01:43.145382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:43.145382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 15:01:43.145382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:43.145382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:43.145382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:43.145382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:43.145382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:43.145382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:43.146415 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:43.146415 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:43.146415 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:43.146415 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:43.146415 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:43.146415 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:43.146415 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 15:01:43.146415 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 15:01:43.146415 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 15:01:43.146415 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 15:01:43.146415 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:43.146415 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:43.146415 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:43.146415 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:43.146415 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:43.146415 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:43.147309 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:43.147309 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:43.147309 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:43.147309 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:43.147309 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:43.147309 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:43.147309 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:43.147309 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:43.147309 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:43.147309 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:43.147309 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 15:01:43.147309 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:01:43.147309 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 27 15:01:43.147309 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:43.152299 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 403/821] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:01:42 2022] GET /identity/v3/services/network => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:43.152891 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b5196-a229-48ef-ac78-bb9bfcbc860d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:43.153016 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4377562d-709f-42d9-b84d-7f90c1e9e6c6 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:43.154539 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b5196-a229-48ef-ac78-bb9bfcbc860d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:43.155268 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b5196-a229-48ef-ac78-bb9bfcbc860d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:43.155773 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066b5196-a229-48ef-ac78-bb9bfcbc860d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:43.169863 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066b5196-a229-48ef-ac78-bb9bfcbc860d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:43.172200 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066b5196-a229-48ef-ac78-bb9bfcbc860d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:43.179379 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:43.191414 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-066b5196-a229-48ef-ac78-bb9bfcbc860d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:43.191414 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:43.191414 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:43.191414 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:43.191414 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:43.191414 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:43.191414 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:43.191414 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:43.191414 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:43.191414 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:43.191414 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:43.191414 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:43.191414 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:43.191414 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:43.191414 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:43.192330 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:43.192330 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:43.192330 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:43.192330 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:43.192330 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:43.192330 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:43.192330 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:43.192330 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:43.192330 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:43.192330 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:43.192330 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:43.192330 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:43.192330 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:43.192330 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:43.192330 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:43.192330 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:43.192330 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:43.193230 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:43.193230 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:43.193230 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:43.193230 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:43.195439 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 419/822] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:43.196422 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b5196-a229-48ef-ac78-bb9bfcbc860d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:43.200368 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:43.267364 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4377562d-709f-42d9-b84d-7f90c1e9e6c6 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:43.303295 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5958f33-ab90-4f6e-ae15-f07bb7f405c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:43.304090 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5958f33-ab90-4f6e-ae15-f07bb7f405c1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:43.304374 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5958f33-ab90-4f6e-ae15-f07bb7f405c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:43.304652 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5958f33-ab90-4f6e-ae15-f07bb7f405c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:43.310689 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066b5196-a229-48ef-ac78-bb9bfcbc860d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:43.311909 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5958f33-ab90-4f6e-ae15-f07bb7f405c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:43.313361 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5958f33-ab90-4f6e-ae15-f07bb7f405c1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:43.327540 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-b5958f33-ab90-4f6e-ae15-f07bb7f405c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:43.327540 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:43.327540 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:43.327540 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:43.327540 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:43.327540 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:43.327540 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:43.327540 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:43.327540 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:43.327540 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:43.327540 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:43.327540 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:43.327540 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 15:01:43.327540 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 15:01:43.327540 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 15:01:43.328465 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 15:01:43.328465 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:43.328465 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:43.328465 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:43.328465 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:43.328465 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:43.328465 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:43.328465 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:43.328465 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:43.328465 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:43.328465 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:43.328465 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:43.328465 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:43.328465 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:43.328465 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:43.328465 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:43.328465 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:43.329363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 15:01:43.329363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 15:01:43.329363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 15:01:43.329363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 15:01:43.329363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 15:01:43.329363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 15:01:43.329363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:43.329363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:43.329363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 404/823] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:01:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:43.329861 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5958f33-ab90-4f6e-ae15-f07bb7f405c1 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:43.348564 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d829ad-5371-4dec-9ebc-3b5314b63edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:43.349236 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d829ad-5371-4dec-9ebc-3b5314b63edf None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:43.349578 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d829ad-5371-4dec-9ebc-3b5314b63edf None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:43.349861 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d829ad-5371-4dec-9ebc-3b5314b63edf None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:43.350620 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d829ad-5371-4dec-9ebc-3b5314b63edf None admin] RBAC: Adding query filter params (name=network) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:43.350957 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d829ad-5371-4dec-9ebc-3b5314b63edf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:43.352324 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d829ad-5371-4dec-9ebc-3b5314b63edf None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:43.362056 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 420/824] 10.222.0.50 () {56 vars in 1112 bytes} [Fri May 27 15:01:43 2022] GET /identity/v3/services?name=network => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:43.363415 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ecdf4-c7d8-4085-ab25-619f31ce2274 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:43.363706 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ecdf4-c7d8-4085-ab25-619f31ce2274 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:43.363987 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ecdf4-c7d8-4085-ab25-619f31ce2274 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:43.365388 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 421/825] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:43.365952 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437ecdf4-c7d8-4085-ab25-619f31ce2274 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:43.403089 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:43.440117 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:43.449283 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5958f33-ab90-4f6e-ae15-f07bb7f405c1 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:43.479428 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437ecdf4-c7d8-4085-ab25-619f31ce2274 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:43.485010 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b358e32e-1ded-48b7-bbdc-874939b52b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:43.485673 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b358e32e-1ded-48b7-bbdc-874939b52b3c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:43.485959 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b358e32e-1ded-48b7-bbdc-874939b52b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:43.486238 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b358e32e-1ded-48b7-bbdc-874939b52b3c None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:43.486983 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b358e32e-1ded-48b7-bbdc-874939b52b3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:43.487296 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b358e32e-1ded-48b7-bbdc-874939b52b3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:43.488656 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b358e32e-1ded-48b7-bbdc-874939b52b3c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:43.504471 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 405/826] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:43.504471 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b358e32e-1ded-48b7-bbdc-874939b52b3c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:43.520310 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423637f9-428b-4192-9631-28a0b2a5eec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:43.521028 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423637f9-428b-4192-9631-28a0b2a5eec9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:43.521311 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423637f9-428b-4192-9631-28a0b2a5eec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:43.521587 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423637f9-428b-4192-9631-28a0b2a5eec9 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:43.522400 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423637f9-428b-4192-9631-28a0b2a5eec9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:43.522709 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423637f9-428b-4192-9631-28a0b2a5eec9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:43.524171 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423637f9-428b-4192-9631-28a0b2a5eec9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:43.537217 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 422/827] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 15:01:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:43.542955 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be533e-48bd-45ba-8e86-812b7cddaee8 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:43.543247 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be533e-48bd-45ba-8e86-812b7cddaee8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:43.543524 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be533e-48bd-45ba-8e86-812b7cddaee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:43.560941 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c2be533e-48bd-45ba-8e86-812b7cddaee8 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:43.562131 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c2be533e-48bd-45ba-8e86-812b7cddaee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:43.609668 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b358e32e-1ded-48b7-bbdc-874939b52b3c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:43.634906 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 423/828] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:01:43.634906 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2be533e-48bd-45ba-8e86-812b7cddaee8 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:43.640273 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:43.651654 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8450c6f-c215-4c4a-9692-76d684a2977b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:43.652517 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8450c6f-c215-4c4a-9692-76d684a2977b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:43.652800 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8450c6f-c215-4c4a-9692-76d684a2977b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:43.653080 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8450c6f-c215-4c4a-9692-76d684a2977b None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:43.653842 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8450c6f-c215-4c4a-9692-76d684a2977b None admin] RBAC: Adding query filter params (type=network) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:43.654094 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8450c6f-c215-4c4a-9692-76d684a2977b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:43.656061 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8450c6f-c215-4c4a-9692-76d684a2977b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:43.661451 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 406/829] 10.222.0.50 () {56 vars in 1112 bytes} [Fri May 27 15:01:43 2022] GET /identity/v3/services?type=network => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:43.662207 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8450c6f-c215-4c4a-9692-76d684a2977b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:43.706273 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:43.750674 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2be533e-48bd-45ba-8e86-812b7cddaee8 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:43.775634 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8450c6f-c215-4c4a-9692-76d684a2977b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:43.787459 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012431c7-911a-4688-97b2-1ba002ac3bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:43.788258 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012431c7-911a-4688-97b2-1ba002ac3bed None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:43.788540 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012431c7-911a-4688-97b2-1ba002ac3bed None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:43.788815 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012431c7-911a-4688-97b2-1ba002ac3bed None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:43.796211 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012431c7-911a-4688-97b2-1ba002ac3bed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:43.798196 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012431c7-911a-4688-97b2-1ba002ac3bed None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:43.811533 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb2952c-a5c5-4e42-ac31-123d5b05a64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:43.812418 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2952c-a5c5-4e42-ac31-123d5b05a64f None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:43.812700 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2952c-a5c5-4e42-ac31-123d5b05a64f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:43.812801 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-012431c7-911a-4688-97b2-1ba002ac3bed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:43.812801 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:43.812801 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:43.812801 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:43.812801 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:43.812801 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:43.812801 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:43.812801 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:43.812801 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:43.812801 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:43.812801 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:43.812801 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:43.812801 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 15:01:43.812801 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 15:01:43.812801 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 15:01:43.813612 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 15:01:43.813612 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:43.813612 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:43.813612 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:43.813612 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:43.813612 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:43.813612 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:43.813612 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:43.813612 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:43.813612 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:43.813612 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:43.813612 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:43.813612 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:43.813612 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:43.813612 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:43.813612 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:43.813612 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:43.814629 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 15:01:43.814629 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 15:01:43.814629 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 15:01:43.814629 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 15:01:43.814629 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 15:01:43.814629 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 15:01:43.814629 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:43.814629 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:43.815622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2952c-a5c5-4e42-ac31-123d5b05a64f None admin] PATH_INFO: `/v3/projects/daf12585428b41a2b45bdbc85413a6c4/users/09c23d1c732c452a9597c647ad0241f0/roles/13f37584013544e7ad1e2c8f350da6d2` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:43.815696 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 424/830] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:01:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:43.815696 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012431c7-911a-4688-97b2-1ba002ac3bed None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:43.832065 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb2952c-a5c5-4e42-ac31-123d5b05a64f None admin] RBAC: Authorizing `identity:create_grant(project_id=daf12585428b41a2b45bdbc85413a6c4, user_id=09c23d1c732c452a9597c647ad0241f0, role_id=13f37584013544e7ad1e2c8f350da6d2)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:43.833521 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb2952c-a5c5-4e42-ac31-123d5b05a64f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:43.849489 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 407/831] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 15:01:43 2022] PUT /identity/v3/projects/daf12585428b41a2b45bdbc85413a6c4/users/09c23d1c732c452a9597c647ad0241f0/roles/13f37584013544e7ad1e2c8f350da6d2 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:01:43.850367 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb2952c-a5c5-4e42-ac31-123d5b05a64f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:43.908366 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:43.955936 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012431c7-911a-4688-97b2-1ba002ac3bed None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:43.977823 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:43.996611 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb2952c-a5c5-4e42-ac31-123d5b05a64f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:44.000829 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c82a9b4-3103-4b6c-b1eb-123ea4939917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:44.002192 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82a9b4-3103-4b6c-b1eb-123ea4939917 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:44.002696 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82a9b4-3103-4b6c-b1eb-123ea4939917 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:44.003199 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82a9b4-3103-4b6c-b1eb-123ea4939917 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:44.019495 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c82a9b4-3103-4b6c-b1eb-123ea4939917 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:01:44.029308 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-3c82a9b4-3103-4b6c-b1eb-123ea4939917 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:01:44.029308 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:44.029308 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:44.029308 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:44.029308 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:44.029308 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:44.029308 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:44.029308 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:44.029308 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:44.029308 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:44.029308 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:44.029308 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:44.029308 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:01:44.029308 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:01:44.029308 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:01:44.029308 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:01:44.030443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:01:44.030443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:01:44.030443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:01:44.030443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:01:44.030443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:44.030443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:44.030443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:44.030443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:44.030443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:44.030443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:44.030443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:44.030443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:44.030443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:44.030443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:44.030443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:44.030443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:44.031335 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:44.031335 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:44.031335 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:44.031335 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:44.031335 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:01:44.031335 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:01:44.031335 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:01:44.031335 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:01:44.031335 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:01:44.031335 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:01:44.031335 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:01:44.031335 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:44.038862 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 425/832] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 15:01:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:44.040106 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c82a9b4-3103-4b6c-b1eb-123ea4939917 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:44.052052 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a52017-c4df-41bd-ae2e-0a8cba381a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:44.054487 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a52017-c4df-41bd-ae2e-0a8cba381a15 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:44.054487 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a52017-c4df-41bd-ae2e-0a8cba381a15 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:44.054979 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a52017-c4df-41bd-ae2e-0a8cba381a15 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:44.056720 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a52017-c4df-41bd-ae2e-0a8cba381a15 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:44.059423 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a52017-c4df-41bd-ae2e-0a8cba381a15 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:44.096304 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 408/833] 10.222.0.50 () {60 vars in 1144 bytes} [Fri May 27 15:01:43 2022] POST /identity/v3/endpoints => generated 327 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:01:44.098610 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a52017-c4df-41bd-ae2e-0a8cba381a15 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:44.209040 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c82a9b4-3103-4b6c-b1eb-123ea4939917 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:44.284658 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:44.303608 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a52017-c4df-41bd-ae2e-0a8cba381a15 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:44.309340 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6644f5-7f85-4cfa-90f7-3f0dd99c25bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:44.310369 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6644f5-7f85-4cfa-90f7-3f0dd99c25bd None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:44.312029 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6644f5-7f85-4cfa-90f7-3f0dd99c25bd None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:44.312569 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6644f5-7f85-4cfa-90f7-3f0dd99c25bd None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:44.313625 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6644f5-7f85-4cfa-90f7-3f0dd99c25bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:44.320246 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6644f5-7f85-4cfa-90f7-3f0dd99c25bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:44.322212 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6644f5-7f85-4cfa-90f7-3f0dd99c25bd None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:44.346868 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 426/834] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 15:01:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:44.349954 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6644f5-7f85-4cfa-90f7-3f0dd99c25bd None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:44.382013 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286f650d-1597-414d-97da-6e0f3d63d855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:44.382623 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286f650d-1597-414d-97da-6e0f3d63d855 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:44.383003 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286f650d-1597-414d-97da-6e0f3d63d855 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:44.383352 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286f650d-1597-414d-97da-6e0f3d63d855 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:44.384202 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286f650d-1597-414d-97da-6e0f3d63d855 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:44.384578 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286f650d-1597-414d-97da-6e0f3d63d855 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:44.386058 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286f650d-1597-414d-97da-6e0f3d63d855 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:44.396735 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 409/835] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:01:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:44.401654 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286f650d-1597-414d-97da-6e0f3d63d855 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:44.413462 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:44.485934 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6644f5-7f85-4cfa-90f7-3f0dd99c25bd None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:44.519187 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286f650d-1597-414d-97da-6e0f3d63d855 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:44.526106 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a4859-2621-4104-a605-37836823bb76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:44.526891 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a4859-2621-4104-a605-37836823bb76 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:44.528680 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a4859-2621-4104-a605-37836823bb76 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:44.529044 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a4859-2621-4104-a605-37836823bb76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:44.530425 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a4859-2621-4104-a605-37836823bb76 None admin] RBAC: Adding query filter params (role.id=759d845eb6284e42931b4265af448ebc, scope.project.id=daf12585428b41a2b45bdbc85413a6c4, user.id=451a9232f95e494683d81b6078a318d0) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:44.530993 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a4859-2621-4104-a605-37836823bb76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:44.532621 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a4859-2621-4104-a605-37836823bb76 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:44.540635 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 427/836] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:01:44 2022] GET /identity/v3/role_assignments?user.id=451a9232f95e494683d81b6078a318d0&scope.project.id=daf12585428b41a2b45bdbc85413a6c4&role.id=759d845eb6284e42931b4265af448ebc => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:01:44.574554 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c84737b-6e3e-4488-8810-2e0254d1abbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:44.575415 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c84737b-6e3e-4488-8810-2e0254d1abbf None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:44.575697 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c84737b-6e3e-4488-8810-2e0254d1abbf None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:44.575971 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c84737b-6e3e-4488-8810-2e0254d1abbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:44.589094 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c84737b-6e3e-4488-8810-2e0254d1abbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:44.590577 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c84737b-6e3e-4488-8810-2e0254d1abbf None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:44.610622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-7c84737b-6e3e-4488-8810-2e0254d1abbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:44.610622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:44.610622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:44.610622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:44.610622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:44.610622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:44.610622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:44.610622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:44.610622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:44.610622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:44.610622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:44.610622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:44.610622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:44.610622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:44.610622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:44.611642 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:44.611642 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:44.611642 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:44.611642 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:44.611642 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:44.611642 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:44.611642 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:44.611642 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:44.611642 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:44.611642 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:44.611642 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:44.611642 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:44.611642 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:44.611642 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:44.611642 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:44.611642 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:44.611642 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:44.612564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:44.612564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:44.612564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:44.612564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:44.612564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 410/837] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:44.617756 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a4859-2621-4104-a605-37836823bb76 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:44.745001 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a4859-2621-4104-a605-37836823bb76 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:44.748087 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:44.789308 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee24e56a-bfdb-4e39-8d1f-df4942ceb212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:44.789980 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee24e56a-bfdb-4e39-8d1f-df4942ceb212 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:44.790264 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee24e56a-bfdb-4e39-8d1f-df4942ceb212 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:44.790541 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee24e56a-bfdb-4e39-8d1f-df4942ceb212 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:44.791305 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee24e56a-bfdb-4e39-8d1f-df4942ceb212 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:44.791613 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee24e56a-bfdb-4e39-8d1f-df4942ceb212 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:44.792971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee24e56a-bfdb-4e39-8d1f-df4942ceb212 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:44.806333 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 428/838] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:44.811579 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c84737b-6e3e-4488-8810-2e0254d1abbf None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:44.816976 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:44.860072 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379701cb-d8cb-466a-a232-329e402eb130 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:44.860399 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379701cb-d8cb-466a-a232-329e402eb130 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:44.860665 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379701cb-d8cb-466a-a232-329e402eb130 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:44.862443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 429/839] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 15:01:44.868273 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ece70-7fd8-4393-961c-6465dcbd7861 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:44.868562 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ece70-7fd8-4393-961c-6465dcbd7861 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:44.868843 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ece70-7fd8-4393-961c-6465dcbd7861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:44.892270 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-906ece70-7fd8-4393-961c-6465dcbd7861 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:44.893508 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-906ece70-7fd8-4393-961c-6465dcbd7861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:44.949835 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c84737b-6e3e-4488-8810-2e0254d1abbf None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:44.971377 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 430/840] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:44 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 15:01:44.995913 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ece70-7fd8-4393-961c-6465dcbd7861 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:44.996908 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3a55ec-579f-4345-86db-35049f39707f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:45.000146 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a55ec-579f-4345-86db-35049f39707f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:45.000668 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a55ec-579f-4345-86db-35049f39707f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:45.001156 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a55ec-579f-4345-86db-35049f39707f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:45.012670 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3a55ec-579f-4345-86db-35049f39707f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:45.015938 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3a55ec-579f-4345-86db-35049f39707f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:45.035989 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-6a3a55ec-579f-4345-86db-35049f39707f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 27 15:01:45.035989 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:45.035989 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:45.035989 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:45.035989 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:45.035989 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:45.035989 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:45.035989 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:45.035989 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:45.035989 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:45.035989 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:45.035989 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:45.035989 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 15:01:45.035989 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 15:01:45.035989 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 15:01:45.035989 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 15:01:45.036993 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:45.036993 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:45.036993 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 15:01:45.036993 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:45.036993 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 15:01:45.036993 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:45.036993 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:45.036993 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:45.036993 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:45.036993 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:45.036993 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:45.036993 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:45.036993 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:45.036993 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:45.036993 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:45.036993 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:45.036993 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:45.036993 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:45.037966 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:45.037966 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:45.037966 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 15:01:45.037966 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 15:01:45.037966 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 15:01:45.037966 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 15:01:45.037966 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 15:01:45.037966 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 15:01:45.037966 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 27 15:01:45.037966 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:45.037966 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 411/841] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 15:01:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:45.044521 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3a55ec-579f-4345-86db-35049f39707f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:45.119009 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:45.127036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:45.131446 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ece70-7fd8-4393-961c-6465dcbd7861 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:45.180117 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddf0250-7ba9-4777-8243-5cbcc46a0d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:45.182209 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf0250-7ba9-4777-8243-5cbcc46a0d06 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:45.182933 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf0250-7ba9-4777-8243-5cbcc46a0d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:45.183402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf0250-7ba9-4777-8243-5cbcc46a0d06 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:45.184293 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddf0250-7ba9-4777-8243-5cbcc46a0d06 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:45.185669 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3a55ec-579f-4345-86db-35049f39707f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:45.186807 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddf0250-7ba9-4777-8243-5cbcc46a0d06 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:45.207120 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-3ddf0250-7ba9-4777-8243-5cbcc46a0d06 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 15:01:45.207120 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:45.207120 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 15:01:45.207120 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 15:01:45.207120 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 15:01:45.207120 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 15:01:45.207120 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 15:01:45.207120 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:01:45.207120 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 15:01:45.207120 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:45.207120 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 15:01:45.207120 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:45.207120 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:45.207120 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:45.207120 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:45.207120 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:45.207120 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:45.208197 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:45.208197 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:45.208197 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:45.208197 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:45.208197 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:45.208197 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:45.208197 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 15:01:45.208197 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 15:01:45.208197 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 15:01:45.208197 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 15:01:45.208197 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:45.208197 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:45.208197 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:45.208197 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:45.208197 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:45.208197 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:45.209076 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:45.209076 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:45.209076 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:45.209076 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:45.209076 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:45.209076 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:45.209076 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:45.209076 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:45.209076 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:45.209076 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:45.209076 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 15:01:45.209076 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:01:45.209076 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 15:01:45.209076 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:45.214017 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 431/842] 10.222.0.50 () {56 vars in 1114 bytes} [Fri May 27 15:01:44 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:01:45.219436 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddf0250-7ba9-4777-8243-5cbcc46a0d06 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:45.253212 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e91722-60ba-4cd5-9b7e-24642c9f90dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:45.254413 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e91722-60ba-4cd5-9b7e-24642c9f90dd None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:45.255031 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e91722-60ba-4cd5-9b7e-24642c9f90dd None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:45.255441 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e91722-60ba-4cd5-9b7e-24642c9f90dd None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:45.256997 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e91722-60ba-4cd5-9b7e-24642c9f90dd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:45.257320 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e91722-60ba-4cd5-9b7e-24642c9f90dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:45.259285 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e91722-60ba-4cd5-9b7e-24642c9f90dd None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:45.284251 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 412/843] 10.222.0.50 () {56 vars in 1135 bytes} [Fri May 27 15:01:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:45.289508 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e91722-60ba-4cd5-9b7e-24642c9f90dd None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:45.344865 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddf0250-7ba9-4777-8243-5cbcc46a0d06 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:45.382267 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfb5814-45c4-4ad6-a70f-7a4bc402afaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:45.383124 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb5814-45c4-4ad6-a70f-7a4bc402afaa None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:45.383576 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb5814-45c4-4ad6-a70f-7a4bc402afaa None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:45.383930 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb5814-45c4-4ad6-a70f-7a4bc402afaa None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:45.384884 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfb5814-45c4-4ad6-a70f-7a4bc402afaa None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:45.385267 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfb5814-45c4-4ad6-a70f-7a4bc402afaa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:45.386708 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfb5814-45c4-4ad6-a70f-7a4bc402afaa None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:45.397379 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 432/844] 10.222.0.50 () {56 vars in 1114 bytes} [Fri May 27 15:01:45 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:45.398384 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfb5814-45c4-4ad6-a70f-7a4bc402afaa None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:45.402957 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e91722-60ba-4cd5-9b7e-24642c9f90dd None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:45.428635 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:45.430806 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:45.443408 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e405dd3-08db-411b-8e25-9f9f4202ef77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:45.444080 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e405dd3-08db-411b-8e25-9f9f4202ef77 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:45.444372 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e405dd3-08db-411b-8e25-9f9f4202ef77 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:45.444657 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e405dd3-08db-411b-8e25-9f9f4202ef77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:45.524762 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e405dd3-08db-411b-8e25-9f9f4202ef77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:45.527135 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfb5814-45c4-4ad6-a70f-7a4bc402afaa None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:45.527317 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e405dd3-08db-411b-8e25-9f9f4202ef77 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:45.564882 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd0014-bf63-4922-b075-7835f485afb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:45.565558 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd0014-bf63-4922-b075-7835f485afb0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:45.565831 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd0014-bf63-4922-b075-7835f485afb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:45.566114 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd0014-bf63-4922-b075-7835f485afb0 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:45.566829 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cd0014-bf63-4922-b075-7835f485afb0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:45.567161 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cd0014-bf63-4922-b075-7835f485afb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:45.568536 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cd0014-bf63-4922-b075-7835f485afb0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:45.583555 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 433/845] 10.222.0.50 () {56 vars in 1114 bytes} [Fri May 27 15:01:45 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:01:45.652163 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 413/846] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:01:45 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 27 15:01:45.658037 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd0014-bf63-4922-b075-7835f485afb0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:45.770575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:45.791411 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd0014-bf63-4922-b075-7835f485afb0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:45.830368 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2718d176-7b99-4652-97ec-97c438d78195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:45.831567 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718d176-7b99-4652-97ec-97c438d78195 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:45.832053 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718d176-7b99-4652-97ec-97c438d78195 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:45.832519 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718d176-7b99-4652-97ec-97c438d78195 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:45.846341 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2718d176-7b99-4652-97ec-97c438d78195 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:45.848143 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2718d176-7b99-4652-97ec-97c438d78195 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:45.876866 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-2718d176-7b99-4652-97ec-97c438d78195 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:45.876866 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:45.876866 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:45.876866 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:45.876866 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:45.876866 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:45.876866 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:45.876866 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:45.876866 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:45.876866 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:45.876866 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:45.876866 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:45.876866 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:45.876866 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:45.876866 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:45.877835 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:45.877835 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:45.877835 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:45.877835 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:45.877835 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:45.877835 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:45.877835 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:45.877835 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:45.877835 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:45.877835 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:45.877835 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:45.877835 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:45.877835 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:45.877835 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:45.877835 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:45.877835 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:45.877835 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:45.878747 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:45.878747 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:45.878747 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:45.878747 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:45.880799 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 434/847] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:45.886507 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e405dd3-08db-411b-8e25-9f9f4202ef77 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:45.911832 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:45.994483 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e405dd3-08db-411b-8e25-9f9f4202ef77 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:46.031809 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf758cb-06a4-45d5-816a-049125ec2f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:46.033202 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf758cb-06a4-45d5-816a-049125ec2f32 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:46.033715 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf758cb-06a4-45d5-816a-049125ec2f32 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:46.034138 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf758cb-06a4-45d5-816a-049125ec2f32 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:46.035028 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf758cb-06a4-45d5-816a-049125ec2f32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:46.035480 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf758cb-06a4-45d5-816a-049125ec2f32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:46.037016 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf758cb-06a4-45d5-816a-049125ec2f32 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:46.050403 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 414/848] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:46.056097 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2718d176-7b99-4652-97ec-97c438d78195 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:46.171496 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2718d176-7b99-4652-97ec-97c438d78195 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:46.206572 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1631d1-f3c7-4e72-a7d6-cc5a4a65ad08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:46.207408 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1631d1-f3c7-4e72-a7d6-cc5a4a65ad08 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:46.207696 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1631d1-f3c7-4e72-a7d6-cc5a4a65ad08 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:46.207977 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1631d1-f3c7-4e72-a7d6-cc5a4a65ad08 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:46.210817 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:46.216548 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1631d1-f3c7-4e72-a7d6-cc5a4a65ad08 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:46.218948 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1631d1-f3c7-4e72-a7d6-cc5a4a65ad08 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:46.234272 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-3a1631d1-f3c7-4e72-a7d6-cc5a4a65ad08 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:46.234272 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:46.234272 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:46.234272 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:46.234272 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:46.234272 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:46.234272 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:46.234272 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:46.234272 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:46.234272 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:46.234272 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:46.234272 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:46.234272 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 15:01:46.234272 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 15:01:46.234272 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 15:01:46.235253 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 15:01:46.235253 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:46.235253 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:46.235253 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:46.235253 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:46.235253 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:46.235253 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:46.235253 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:46.235253 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:46.235253 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:46.235253 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:46.235253 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:46.235253 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:46.235253 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:46.235253 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:46.235253 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:46.235253 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:46.236356 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 15:01:46.236356 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 15:01:46.236356 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 15:01:46.236356 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 15:01:46.236356 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 15:01:46.236356 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 15:01:46.236356 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:46.236356 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:46.236356 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 435/849] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:01:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:46.242938 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf758cb-06a4-45d5-816a-049125ec2f32 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:46.331352 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:46.352403 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf758cb-06a4-45d5-816a-049125ec2f32 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:46.394446 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f447c763-5cd1-45f6-9536-3df3b0be59bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:46.395594 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447c763-5cd1-45f6-9536-3df3b0be59bd None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:46.396130 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447c763-5cd1-45f6-9536-3df3b0be59bd None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:46.396610 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f447c763-5cd1-45f6-9536-3df3b0be59bd None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:46.397612 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f447c763-5cd1-45f6-9536-3df3b0be59bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:46.398137 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f447c763-5cd1-45f6-9536-3df3b0be59bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:46.399871 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f447c763-5cd1-45f6-9536-3df3b0be59bd None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:46.414198 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 415/850] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 15:01:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:46.424034 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1631d1-f3c7-4e72-a7d6-cc5a4a65ad08 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:46.538522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1631d1-f3c7-4e72-a7d6-cc5a4a65ad08 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:46.576180 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4a717b-1321-420c-8d8f-02b8d354f744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:46.577868 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a717b-1321-420c-8d8f-02b8d354f744 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:46.578358 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a717b-1321-420c-8d8f-02b8d354f744 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:46.578834 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a717b-1321-420c-8d8f-02b8d354f744 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:46.579901 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4a717b-1321-420c-8d8f-02b8d354f744 None admin] RBAC: Adding query filter params (role.id=759d845eb6284e42931b4265af448ebc, scope.project.id=daf12585428b41a2b45bdbc85413a6c4, user.id=18903f2eab7c415096e075d47f0b9a4d) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:46.580432 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4a717b-1321-420c-8d8f-02b8d354f744 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:46.582145 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4a717b-1321-420c-8d8f-02b8d354f744 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:46.590000 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 436/851] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:01:46 2022] GET /identity/v3/role_assignments?user.id=18903f2eab7c415096e075d47f0b9a4d&scope.project.id=daf12585428b41a2b45bdbc85413a6c4&role.id=759d845eb6284e42931b4265af448ebc => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:46.707077 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb69c58-30f8-4657-8070-2d1849402215 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:46.707746 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb69c58-30f8-4657-8070-2d1849402215 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:46.708310 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb69c58-30f8-4657-8070-2d1849402215 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:46.710189 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 416/852] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:46.715882 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e15f9-d6d5-4800-a5b3-74191ca82f3f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:46.716171 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e15f9-d6d5-4800-a5b3-74191ca82f3f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:46.716450 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5e15f9-d6d5-4800-a5b3-74191ca82f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:46.735812 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3c5e15f9-d6d5-4800-a5b3-74191ca82f3f None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:46.737048 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3c5e15f9-d6d5-4800-a5b3-74191ca82f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:46.809599 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:46.824004 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 437/853] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:46 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:01:46.851205 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb69c58-30f8-4657-8070-2d1849402215 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:46.977962 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb69c58-30f8-4657-8070-2d1849402215 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:46.979830 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec2b9ae-819b-4596-b32c-9d3a7e96ced9 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:46.980121 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec2b9ae-819b-4596-b32c-9d3a7e96ced9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:46.980402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec2b9ae-819b-4596-b32c-9d3a7e96ced9 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:46.981904 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 438/854] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:46.987432 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577be015-3e8c-4010-874c-92e9be59067a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:46.987718 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577be015-3e8c-4010-874c-92e9be59067a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:46.987997 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577be015-3e8c-4010-874c-92e9be59067a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:47.006026 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-577be015-3e8c-4010-874c-92e9be59067a None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:47.007238 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-577be015-3e8c-4010-874c-92e9be59067a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:47.031508 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4030221-875d-46c0-a6fc-653b61c13b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:47.032458 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4030221-875d-46c0-a6fc-653b61c13b75 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:47.033253 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4030221-875d-46c0-a6fc-653b61c13b75 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:47.033253 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4030221-875d-46c0-a6fc-653b61c13b75 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:47.048291 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4030221-875d-46c0-a6fc-653b61c13b75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:01:47.057383 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-f4030221-875d-46c0-a6fc-653b61c13b75 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 15:01:47.057383 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:47.057383 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:47.057383 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:47.057383 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:47.057383 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:47.057383 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:47.057383 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:47.057383 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:47.057383 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:47.057383 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:47.057383 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:47.057383 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:01:47.057383 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:01:47.057383 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:01:47.057383 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:01:47.059073 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:01:47.059073 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:01:47.059073 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:01:47.059073 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:01:47.059073 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:47.059073 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:47.059073 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:47.059073 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:47.059073 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:47.059073 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:47.059073 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:47.059073 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:47.059073 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:47.059073 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:47.059073 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:47.059073 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:47.060506 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:47.060506 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:47.060506 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:47.060506 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:47.060506 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:01:47.060506 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:01:47.060506 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:01:47.060506 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:01:47.060506 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:01:47.060506 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:01:47.060506 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 15:01:47.060506 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:47.060506 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 417/855] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 15:01:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:47.065177 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4030221-875d-46c0-a6fc-653b61c13b75 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:47.067652 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:47.107120 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 439/856] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:46 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 15:01:47.122536 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577be015-3e8c-4010-874c-92e9be59067a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:47.176563 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4030221-875d-46c0-a6fc-653b61c13b75 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:47.212290 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca7fc4-5011-4eba-926c-d6374407edc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:47.213058 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca7fc4-5011-4eba-926c-d6374407edc4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:47.213638 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca7fc4-5011-4eba-926c-d6374407edc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:47.213914 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca7fc4-5011-4eba-926c-d6374407edc4 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:47.214669 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eca7fc4-5011-4eba-926c-d6374407edc4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:47.215010 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eca7fc4-5011-4eba-926c-d6374407edc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:47.216458 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eca7fc4-5011-4eba-926c-d6374407edc4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:47.226400 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 418/857] 10.222.0.50 () {56 vars in 1103 bytes} [Fri May 27 15:01:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:47.231838 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca7fc4-5011-4eba-926c-d6374407edc4 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:47.261949 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577be015-3e8c-4010-874c-92e9be59067a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:47.292017 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:47.310295 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:47.312898 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7df44a-0c5e-4886-a14e-44782dfd8619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:47.313926 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7df44a-0c5e-4886-a14e-44782dfd8619 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:47.314558 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7df44a-0c5e-4886-a14e-44782dfd8619 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:47.315040 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7df44a-0c5e-4886-a14e-44782dfd8619 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:47.316440 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7df44a-0c5e-4886-a14e-44782dfd8619 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:47.319123 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7df44a-0c5e-4886-a14e-44782dfd8619 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:47.334266 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-7e7df44a-0c5e-4886-a14e-44782dfd8619 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 15:01:47.334266 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:47.334266 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 15:01:47.334266 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 15:01:47.334266 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 15:01:47.334266 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 15:01:47.334266 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 15:01:47.334266 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:01:47.334266 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 15:01:47.334266 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:47.334266 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 15:01:47.334266 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:47.334266 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:47.334266 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:47.334266 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:47.334266 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:47.335302 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:47.335302 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:47.335302 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:47.335302 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:47.335302 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:47.335302 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:47.335302 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:47.335302 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 15:01:47.335302 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 15:01:47.335302 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 15:01:47.335302 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 15:01:47.335302 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:47.335302 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:47.335302 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:47.335302 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:47.335302 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:47.336157 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:47.336157 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:47.336157 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:47.336157 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:47.336157 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:47.336157 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:47.336157 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:47.336157 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:47.336157 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:47.336157 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:47.336157 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:47.336157 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 15:01:47.336157 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:01:47.336157 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 15:01:47.336157 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:47.336157 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 440/858] 10.222.0.50 () {56 vars in 1132 bytes} [Fri May 27 15:01:47 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:01:47.341410 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7df44a-0c5e-4886-a14e-44782dfd8619 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:47.363600 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca7fc4-5011-4eba-926c-d6374407edc4 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:47.401823 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d5e176-c325-4134-8f3a-f39ea2b3fabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:47.403087 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5e176-c325-4134-8f3a-f39ea2b3fabe None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:47.403670 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5e176-c325-4134-8f3a-f39ea2b3fabe None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:47.404149 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5e176-c325-4134-8f3a-f39ea2b3fabe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:47.417177 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d5e176-c325-4134-8f3a-f39ea2b3fabe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:47.419140 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d5e176-c325-4134-8f3a-f39ea2b3fabe None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:47.443024 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-49d5e176-c325-4134-8f3a-f39ea2b3fabe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:47.443024 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:47.443024 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:47.443024 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:47.443024 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:47.443024 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:47.443024 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:47.443024 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:47.443024 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:47.443024 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:47.443024 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:47.443024 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:47.443024 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:47.443024 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:47.443024 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:47.444001 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:47.444001 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:47.444001 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:47.444001 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:47.444001 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:47.444001 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:47.444001 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:47.444001 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:47.444001 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:47.444001 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:47.444001 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:47.444001 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:47.444001 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:47.444001 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:47.444001 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:47.444001 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:47.444001 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:47.444902 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:47.444902 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:47.444902 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:47.444902 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:47.444902 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 419/859] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:01:47.449629 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d5e176-c325-4134-8f3a-f39ea2b3fabe None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:47.472944 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7df44a-0c5e-4886-a14e-44782dfd8619 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:47.516063 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e357cd8-0cb9-468c-806f-5d628bc381bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:47.516769 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e357cd8-0cb9-468c-806f-5d628bc381bd None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:47.517049 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e357cd8-0cb9-468c-806f-5d628bc381bd None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:47.517319 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e357cd8-0cb9-468c-806f-5d628bc381bd None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:47.518027 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e357cd8-0cb9-468c-806f-5d628bc381bd None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:47.518335 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e357cd8-0cb9-468c-806f-5d628bc381bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:47.519769 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e357cd8-0cb9-468c-806f-5d628bc381bd None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:47.525084 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 441/860] 10.222.0.50 () {56 vars in 1126 bytes} [Fri May 27 15:01:47 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:01:47.532397 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e357cd8-0cb9-468c-806f-5d628bc381bd None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:47.582252 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d5e176-c325-4134-8f3a-f39ea2b3fabe None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:47.600732 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:47.615293 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:47.621137 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beda9cd-81db-4431-8701-dbbeb2e6518e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:47.621813 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beda9cd-81db-4431-8701-dbbeb2e6518e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:47.622168 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beda9cd-81db-4431-8701-dbbeb2e6518e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:47.622400 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beda9cd-81db-4431-8701-dbbeb2e6518e None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:47.623276 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beda9cd-81db-4431-8701-dbbeb2e6518e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:47.624609 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beda9cd-81db-4431-8701-dbbeb2e6518e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:47.625719 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beda9cd-81db-4431-8701-dbbeb2e6518e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:47.639587 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 420/861] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:47.645092 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beda9cd-81db-4431-8701-dbbeb2e6518e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:47.677328 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e357cd8-0cb9-468c-806f-5d628bc381bd None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:47.723927 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d75baf-3956-41ce-8e6c-6cfa4b18088f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:47.725006 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d75baf-3956-41ce-8e6c-6cfa4b18088f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:47.725496 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d75baf-3956-41ce-8e6c-6cfa4b18088f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:47.725976 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d75baf-3956-41ce-8e6c-6cfa4b18088f None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:47.726900 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d75baf-3956-41ce-8e6c-6cfa4b18088f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:47.727412 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d75baf-3956-41ce-8e6c-6cfa4b18088f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:47.729574 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d75baf-3956-41ce-8e6c-6cfa4b18088f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:47.737081 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 442/862] 10.222.0.50 () {56 vars in 1126 bytes} [Fri May 27 15:01:47 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:47.764909 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beda9cd-81db-4431-8701-dbbeb2e6518e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:47.810004 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd91aeab-f722-4997-94a8-3a435e2f35fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:47.816067 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91aeab-f722-4997-94a8-3a435e2f35fa None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:47.816714 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91aeab-f722-4997-94a8-3a435e2f35fa None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:47.817196 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91aeab-f722-4997-94a8-3a435e2f35fa None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:47.827535 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd91aeab-f722-4997-94a8-3a435e2f35fa None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:47.829383 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd91aeab-f722-4997-94a8-3a435e2f35fa None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:47.847867 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-cd91aeab-f722-4997-94a8-3a435e2f35fa None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 15:01:47.847867 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:47.847867 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:47.847867 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:47.847867 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:47.847867 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:47.847867 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:47.847867 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:47.847867 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:47.847867 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:47.847867 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:47.847867 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:47.847867 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 15:01:47.847867 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 15:01:47.847867 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 15:01:47.847867 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 15:01:47.849521 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:47.849521 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:47.849521 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 15:01:47.849521 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:47.849521 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 15:01:47.849521 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:47.849521 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:47.849521 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:47.849521 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:47.849521 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:47.849521 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:47.849521 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:47.849521 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:47.849521 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:47.849521 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:47.849521 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:47.849521 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:47.849521 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:47.850478 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:47.850478 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:47.850478 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 15:01:47.850478 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 15:01:47.850478 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 15:01:47.850478 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 15:01:47.850478 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 15:01:47.850478 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 15:01:47.850478 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 15:01:47.850478 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:47.853274 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 421/863] 10.222.0.50 () {56 vars in 1117 bytes} [Fri May 27 15:01:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:01:47.860333 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d75baf-3956-41ce-8e6c-6cfa4b18088f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:47.981312 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d75baf-3956-41ce-8e6c-6cfa4b18088f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:47.998354 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:48.023955 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1ba837-7647-42a8-9944-66c416074645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:48.024641 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ba837-7647-42a8-9944-66c416074645 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:48.024924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ba837-7647-42a8-9944-66c416074645 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:48.025204 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1ba837-7647-42a8-9944-66c416074645 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:48.026053 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1ba837-7647-42a8-9944-66c416074645 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:48.026361 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1ba837-7647-42a8-9944-66c416074645 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:48.027874 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1ba837-7647-42a8-9944-66c416074645 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:48.047494 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 443/864] 10.222.0.50 () {56 vars in 1149 bytes} [Fri May 27 15:01:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:48.053047 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd91aeab-f722-4997-94a8-3a435e2f35fa None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:48.092622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:48.168587 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c25dc-1355-4a3b-ae6e-b8163c856337 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:48.168892 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c25dc-1355-4a3b-ae6e-b8163c856337 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:48.169199 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c25dc-1355-4a3b-ae6e-b8163c856337 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:48.172296 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 444/865] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:48.174615 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd91aeab-f722-4997-94a8-3a435e2f35fa None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:48.178988 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5b83c-3897-426d-a5f6-e449dcfdb470 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:48.179503 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5b83c-3897-426d-a5f6-e449dcfdb470 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:48.179927 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5b83c-3897-426d-a5f6-e449dcfdb470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:48.198794 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1fd5b83c-3897-426d-a5f6-e449dcfdb470 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:48.200021 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1fd5b83c-3897-426d-a5f6-e449dcfdb470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:48.233951 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ad73fb-c6c6-4495-8c53-84122c7d2e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:48.235136 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad73fb-c6c6-4495-8c53-84122c7d2e02 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:48.235666 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad73fb-c6c6-4495-8c53-84122c7d2e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:48.235945 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ad73fb-c6c6-4495-8c53-84122c7d2e02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:48.279127 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 445/866] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:48 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:01:48.295794 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5b83c-3897-426d-a5f6-e449dcfdb470 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:48.319605 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ad73fb-c6c6-4495-8c53-84122c7d2e02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:48.321552 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ad73fb-c6c6-4495-8c53-84122c7d2e02 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:48.366662 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:48.397877 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:48.403161 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5b83c-3897-426d-a5f6-e449dcfdb470 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:48.443133 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c436782-3b9c-4c39-84b1-1dfa09b44018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:48.444199 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c436782-3b9c-4c39-84b1-1dfa09b44018 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:48.444692 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c436782-3b9c-4c39-84b1-1dfa09b44018 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:48.445173 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c436782-3b9c-4c39-84b1-1dfa09b44018 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:48.446276 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c436782-3b9c-4c39-84b1-1dfa09b44018 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:48.447915 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c436782-3b9c-4c39-84b1-1dfa09b44018 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:48.453788 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 422/867] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:01:48 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 15:01:48.459648 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ad73fb-c6c6-4495-8c53-84122c7d2e02 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:48.471273 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 446/868] 10.222.0.50 () {60 vars in 1141 bytes} [Fri May 27 15:01:48 2022] POST /identity/v3/services => generated 251 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:01:48.575937 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ad73fb-c6c6-4495-8c53-84122c7d2e02 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:48.622378 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:48.628043 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef8fa2b-da16-4069-a0fb-8b2bc499e724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:48.628857 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8fa2b-da16-4069-a0fb-8b2bc499e724 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:48.629139 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8fa2b-da16-4069-a0fb-8b2bc499e724 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:48.629416 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef8fa2b-da16-4069-a0fb-8b2bc499e724 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:48.641757 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef8fa2b-da16-4069-a0fb-8b2bc499e724 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:48.643275 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef8fa2b-da16-4069-a0fb-8b2bc499e724 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:48.663094 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-fef8fa2b-da16-4069-a0fb-8b2bc499e724 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:48.663094 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:48.663094 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:48.663094 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:48.663094 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:48.663094 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:48.663094 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:48.663094 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:48.663094 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:48.663094 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:48.663094 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:48.663094 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:48.663094 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:48.663094 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:48.663094 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:48.665213 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:48.665213 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:48.665213 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:48.665213 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:48.665213 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:48.665213 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:48.665213 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:48.665213 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:48.665213 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:48.665213 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:48.665213 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:48.665213 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:48.665213 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:48.665213 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:48.665213 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:48.665213 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:48.665213 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:48.666436 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:48.666436 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:48.666436 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:48.666436 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:48.666436 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 423/869] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:01:48.670050 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c436782-3b9c-4c39-84b1-1dfa09b44018 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:48.790499 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c436782-3b9c-4c39-84b1-1dfa09b44018 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:48.830428 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d0129c-2468-4e79-8e13-2aef371db4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:48.831533 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d0129c-2468-4e79-8e13-2aef371db4fe None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:48.832027 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d0129c-2468-4e79-8e13-2aef371db4fe None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:48.832512 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d0129c-2468-4e79-8e13-2aef371db4fe None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:48.833432 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d0129c-2468-4e79-8e13-2aef371db4fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:48.833940 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d0129c-2468-4e79-8e13-2aef371db4fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:48.835535 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d0129c-2468-4e79-8e13-2aef371db4fe None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:48.850598 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 447/870] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:48.855866 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef8fa2b-da16-4069-a0fb-8b2bc499e724 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:48.990779 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef8fa2b-da16-4069-a0fb-8b2bc499e724 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:49.036172 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8422c175-decf-4727-b562-0440420ae5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:49.037339 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422c175-decf-4727-b562-0440420ae5ff None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:49.037898 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422c175-decf-4727-b562-0440420ae5ff None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:49.038372 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8422c175-decf-4727-b562-0440420ae5ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:49.046110 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8422c175-decf-4727-b562-0440420ae5ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:49.047949 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8422c175-decf-4727-b562-0440420ae5ff None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:49.063207 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-8422c175-decf-4727-b562-0440420ae5ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:49.063207 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:49.063207 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:49.063207 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:49.063207 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:49.063207 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:49.063207 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:49.063207 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:49.063207 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:49.063207 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:49.063207 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:49.063207 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:49.063207 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 15:01:49.063207 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 15:01:49.063207 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 15:01:49.064637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 15:01:49.064637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:49.064637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:49.064637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:49.064637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:49.064637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:49.064637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:49.064637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:49.064637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:49.064637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:49.064637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:49.064637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:49.064637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:49.064637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:49.064637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:49.064637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:49.064637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:49.065582 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 15:01:49.065582 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 15:01:49.065582 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 15:01:49.065582 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 15:01:49.065582 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 15:01:49.065582 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 15:01:49.065582 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:49.065582 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:49.066033 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 424/871] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:01:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:49.072502 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d0129c-2468-4e79-8e13-2aef371db4fe None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:49.099478 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:49.132774 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2d264-e719-4316-a106-801bf2459060 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:49.133153 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2d264-e719-4316-a106-801bf2459060 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:49.133344 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd2d264-e719-4316-a106-801bf2459060 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:49.134924 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 425/872] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 15:01:49.141681 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0131d-901f-46eb-a445-dfaeca2cf82f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:49.142301 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0131d-901f-46eb-a445-dfaeca2cf82f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:49.142790 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f0131d-901f-46eb-a445-dfaeca2cf82f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:49.174875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-29f0131d-901f-46eb-a445-dfaeca2cf82f None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:49.174875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-29f0131d-901f-46eb-a445-dfaeca2cf82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:49.220944 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d0129c-2468-4e79-8e13-2aef371db4fe None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:49.222009 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:49.263588 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 426/873] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:49 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:01:49.271679 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1de4df0-ae47-41ba-909e-48e5be63e709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:49.272584 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1de4df0-ae47-41ba-909e-48e5be63e709 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:49.273296 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1de4df0-ae47-41ba-909e-48e5be63e709 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:49.273928 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1de4df0-ae47-41ba-909e-48e5be63e709 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:49.275368 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1de4df0-ae47-41ba-909e-48e5be63e709 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:49.275852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1de4df0-ae47-41ba-909e-48e5be63e709 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:49.277454 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1de4df0-ae47-41ba-909e-48e5be63e709 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:49.286184 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f0131d-901f-46eb-a445-dfaeca2cf82f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:49.293295 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 448/874] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 15:01:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:49.300459 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1de4df0-ae47-41ba-909e-48e5be63e709 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:49.396795 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:49.410629 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f0131d-901f-46eb-a445-dfaeca2cf82f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:49.421961 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1de4df0-ae47-41ba-909e-48e5be63e709 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:49.462131 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90569af8-8001-4ff5-9419-93fe540439fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:49.462971 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90569af8-8001-4ff5-9419-93fe540439fe None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:49.463495 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcb0fc6-508e-4f18-870d-d06ee7c6155e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:49.463633 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90569af8-8001-4ff5-9419-93fe540439fe None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:49.463776 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90569af8-8001-4ff5-9419-93fe540439fe None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:49.464438 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90569af8-8001-4ff5-9419-93fe540439fe None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:49.464721 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb0fc6-508e-4f18-870d-d06ee7c6155e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:49.465147 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb0fc6-508e-4f18-870d-d06ee7c6155e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:49.465575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb0fc6-508e-4f18-870d-d06ee7c6155e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:49.465809 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90569af8-8001-4ff5-9419-93fe540439fe None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:49.466730 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcb0fc6-508e-4f18-870d-d06ee7c6155e None admin] RBAC: Adding query filter params (role.id=13f37584013544e7ad1e2c8f350da6d2, scope.project.id=daf12585428b41a2b45bdbc85413a6c4, user.id=09c23d1c732c452a9597c647ad0241f0) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:49.467283 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcb0fc6-508e-4f18-870d-d06ee7c6155e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:49.468889 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcb0fc6-508e-4f18-870d-d06ee7c6155e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:49.483157 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-90569af8-8001-4ff5-9419-93fe540439fe None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 15:01:49.483157 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:49.483157 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 15:01:49.483157 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 15:01:49.483157 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 15:01:49.483157 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 15:01:49.483157 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 15:01:49.483157 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:01:49.483157 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 15:01:49.483157 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:49.483157 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 15:01:49.483157 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:49.483157 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:49.483157 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:49.483157 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:49.483157 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:49.483157 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:49.484335 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:49.484335 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:49.484335 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:49.484335 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:49.484335 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:49.484335 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:49.484335 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 15:01:49.484335 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 15:01:49.484335 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 15:01:49.484335 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 15:01:49.484335 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:49.484335 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:49.484335 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:49.484335 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:49.484335 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:49.484335 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:49.485208 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:49.485208 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:49.485208 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:49.485208 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:49.485208 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:49.485208 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:49.485208 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:49.485208 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:49.485208 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:49.485208 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:49.485208 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 15:01:49.485208 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:01:49.485208 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 15:01:49.485208 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:49.486617 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 427/875] 10.222.0.50 () {56 vars in 1126 bytes} [Fri May 27 15:01:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:01:49.486986 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 449/876] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:01:49 2022] GET /identity/v3/role_assignments?user.id=09c23d1c732c452a9597c647ad0241f0&scope.project.id=daf12585428b41a2b45bdbc85413a6c4&role.id=13f37584013544e7ad1e2c8f350da6d2 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:01:49.488207 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90569af8-8001-4ff5-9419-93fe540439fe None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:49.535806 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:49.606204 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90569af8-8001-4ff5-9419-93fe540439fe None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:49.651926 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba49b16-47aa-4c29-b0e1-4296c2bde2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:49.652597 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba49b16-47aa-4c29-b0e1-4296c2bde2b0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:49.653260 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba49b16-47aa-4c29-b0e1-4296c2bde2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:49.653260 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba49b16-47aa-4c29-b0e1-4296c2bde2b0 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:49.653968 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba49b16-47aa-4c29-b0e1-4296c2bde2b0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:49.654279 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba49b16-47aa-4c29-b0e1-4296c2bde2b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:49.655755 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba49b16-47aa-4c29-b0e1-4296c2bde2b0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:49.666576 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 428/877] 10.222.0.50 () {56 vars in 1122 bytes} [Fri May 27 15:01:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:49.672245 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcb0fc6-508e-4f18-870d-d06ee7c6155e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:49.800844 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcb0fc6-508e-4f18-870d-d06ee7c6155e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:49.839033 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c484f8f1-1791-4444-82a3-db477b083eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:49.839712 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c484f8f1-1791-4444-82a3-db477b083eca None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:49.839997 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c484f8f1-1791-4444-82a3-db477b083eca None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:49.840281 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c484f8f1-1791-4444-82a3-db477b083eca None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:49.840999 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c484f8f1-1791-4444-82a3-db477b083eca None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:49.841304 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c484f8f1-1791-4444-82a3-db477b083eca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:49.843321 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c484f8f1-1791-4444-82a3-db477b083eca None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:49.849396 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 450/878] 10.222.0.50 () {56 vars in 1122 bytes} [Fri May 27 15:01:49 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:50.288347 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffde22d-4551-422a-b220-911b81e52ed4 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:50.288909 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffde22d-4551-422a-b220-911b81e52ed4 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:50.289178 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffde22d-4551-422a-b220-911b81e52ed4 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:50.291338 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 429/879] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:50.297847 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31766926-ed9e-47c3-97a7-01b0e00641c3 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:50.298285 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31766926-ed9e-47c3-97a7-01b0e00641c3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:50.298491 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31766926-ed9e-47c3-97a7-01b0e00641c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:50.317638 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-31766926-ed9e-47c3-97a7-01b0e00641c3 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:50.318838 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-31766926-ed9e-47c3-97a7-01b0e00641c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:50.321370 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:50.398265 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 451/880] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:50 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 15:01:50.425957 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffde22d-4551-422a-b220-911b81e52ed4 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:50.546933 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffde22d-4551-422a-b220-911b81e52ed4 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:50.548541 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:50.585184 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8dc86b-a3b8-4121-b3b0-ffe758476496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:50.585853 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8dc86b-a3b8-4121-b3b0-ffe758476496 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:50.586137 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8dc86b-a3b8-4121-b3b0-ffe758476496 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:50.586408 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8dc86b-a3b8-4121-b3b0-ffe758476496 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:50.587182 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8dc86b-a3b8-4121-b3b0-ffe758476496 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:50.588980 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8dc86b-a3b8-4121-b3b0-ffe758476496 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:50.606799 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 430/881] 10.222.0.50 () {60 vars in 1141 bytes} [Fri May 27 15:01:50 2022] POST /identity/v3/services => generated 269 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:01:50.932773 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8c259-b9be-4ed1-9c97-02ec0cc79777 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:50.933035 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8c259-b9be-4ed1-9c97-02ec0cc79777 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:50.933315 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8c259-b9be-4ed1-9c97-02ec0cc79777 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:50.935014 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 452/882] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:50.941037 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef177e5-cb0a-4fe7-8ead-2f62f204978a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:50.941327 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef177e5-cb0a-4fe7-8ead-2f62f204978a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:50.941591 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef177e5-cb0a-4fe7-8ead-2f62f204978a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:50.959439 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3ef177e5-cb0a-4fe7-8ead-2f62f204978a None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:50.960784 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3ef177e5-cb0a-4fe7-8ead-2f62f204978a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:51.049909 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 431/883] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:50 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:01:51.072417 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8c259-b9be-4ed1-9c97-02ec0cc79777 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:51.212949 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef8c259-b9be-4ed1-9c97-02ec0cc79777 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:51.250343 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf30e8e8-a7e6-4d9d-acef-92092ca734d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:51.251430 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf30e8e8-a7e6-4d9d-acef-92092ca734d0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:51.251890 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf30e8e8-a7e6-4d9d-acef-92092ca734d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:51.252324 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf30e8e8-a7e6-4d9d-acef-92092ca734d0 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:51.253133 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf30e8e8-a7e6-4d9d-acef-92092ca734d0 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:51.254648 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf30e8e8-a7e6-4d9d-acef-92092ca734d0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:51.267030 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-cf30e8e8-a7e6-4d9d-acef-92092ca734d0 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 15:01:51.267030 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:51.267030 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 15:01:51.267030 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 15:01:51.267030 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 15:01:51.267030 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 15:01:51.267030 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 15:01:51.267030 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:01:51.267030 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 15:01:51.267030 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:51.267030 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 15:01:51.267030 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:51.267030 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:51.267030 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:51.267030 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:51.267030 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:51.267030 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:51.268264 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:51.268264 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:51.268264 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:51.268264 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:51.268264 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:51.268264 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:51.268264 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 15:01:51.268264 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 15:01:51.268264 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 15:01:51.268264 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 15:01:51.268264 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:51.268264 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:51.268264 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:51.268264 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:51.268264 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:51.268264 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:51.269343 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:51.269343 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:51.269343 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:51.269343 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:51.269343 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:51.269343 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:51.269343 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:51.269343 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:51.269343 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:51.269343 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:51.269343 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 15:01:51.269343 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:01:51.269343 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 15:01:51.269343 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:51.271782 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 453/884] 10.222.0.50 () {56 vars in 1114 bytes} [Fri May 27 15:01:51 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:51.277423 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef177e5-cb0a-4fe7-8ead-2f62f204978a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:51.371302 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:51.383240 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef177e5-cb0a-4fe7-8ead-2f62f204978a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:51.424080 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2dcbd9-f64c-4fb4-984b-50ad2decd38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:51.424758 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2dcbd9-f64c-4fb4-984b-50ad2decd38e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:51.425039 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2dcbd9-f64c-4fb4-984b-50ad2decd38e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:51.425306 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2dcbd9-f64c-4fb4-984b-50ad2decd38e None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:51.426022 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2dcbd9-f64c-4fb4-984b-50ad2decd38e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:51.426326 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2dcbd9-f64c-4fb4-984b-50ad2decd38e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:51.427724 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2dcbd9-f64c-4fb4-984b-50ad2decd38e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:51.433368 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 432/885] 10.222.0.50 () {56 vars in 1114 bytes} [Fri May 27 15:01:51 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:51.438659 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf30e8e8-a7e6-4d9d-acef-92092ca734d0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:51.442338 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:51.556868 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf30e8e8-a7e6-4d9d-acef-92092ca734d0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:51.595783 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4d79a6-81e5-4c0e-b770-e8a90556ef10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:51.596785 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d79a6-81e5-4c0e-b770-e8a90556ef10 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:51.597254 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d79a6-81e5-4c0e-b770-e8a90556ef10 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:51.597731 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4d79a6-81e5-4c0e-b770-e8a90556ef10 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:51.598632 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4d79a6-81e5-4c0e-b770-e8a90556ef10 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:51.599151 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4d79a6-81e5-4c0e-b770-e8a90556ef10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:51.600696 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4d79a6-81e5-4c0e-b770-e8a90556ef10 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:51.606318 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 454/886] 10.222.0.50 () {56 vars in 1114 bytes} [Fri May 27 15:01:51 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:51.611939 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2dcbd9-f64c-4fb4-984b-50ad2decd38e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:51.719326 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2dcbd9-f64c-4fb4-984b-50ad2decd38e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:51.760831 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2586f0e-761a-48ac-8699-aa6a36e741c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:51.761599 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2586f0e-761a-48ac-8699-aa6a36e741c3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:51.762040 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2586f0e-761a-48ac-8699-aa6a36e741c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:51.762528 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2586f0e-761a-48ac-8699-aa6a36e741c3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:51.764473 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2586f0e-761a-48ac-8699-aa6a36e741c3 None admin] RBAC: Adding query filter params (interface=public, service_id=bf6cb01b43c240459987ce5baf5aebc1, region_id=RegionOne) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:51.765027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2586f0e-761a-48ac-8699-aa6a36e741c3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:51.767373 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2586f0e-761a-48ac-8699-aa6a36e741c3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:51.773969 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 433/887] 10.222.0.50 () {56 vars in 1251 bytes} [Fri May 27 15:01:51 2022] GET /identity/v3/endpoints?service_id=bf6cb01b43c240459987ce5baf5aebc1&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:51.780334 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4d79a6-81e5-4c0e-b770-e8a90556ef10 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:51.858766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:51.901558 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4d79a6-81e5-4c0e-b770-e8a90556ef10 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:51.939221 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07cd131-3524-4b68-bca1-7d9e8c1202a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:51.939908 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07cd131-3524-4b68-bca1-7d9e8c1202a1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:51.940189 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07cd131-3524-4b68-bca1-7d9e8c1202a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:51.940463 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07cd131-3524-4b68-bca1-7d9e8c1202a1 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:51.941151 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07cd131-3524-4b68-bca1-7d9e8c1202a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:51.942560 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07cd131-3524-4b68-bca1-7d9e8c1202a1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:51.949658 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 455/888] 10.222.0.50 () {56 vars in 1087 bytes} [Fri May 27 15:01:51 2022] GET /identity/v3/services => generated 1268 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:01:52.142624 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb099995-13fd-49c8-b2f7-dc0cc9cbb244 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:52.142987 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb099995-13fd-49c8-b2f7-dc0cc9cbb244 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:52.143204 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb099995-13fd-49c8-b2f7-dc0cc9cbb244 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:52.144905 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 434/889] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:52.151004 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74047870-64ab-4703-b249-00b68854f075 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:52.151299 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74047870-64ab-4703-b249-00b68854f075 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:52.151579 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74047870-64ab-4703-b249-00b68854f075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:52.175345 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-74047870-64ab-4703-b249-00b68854f075 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:52.176533 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-74047870-64ab-4703-b249-00b68854f075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:52.254195 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 456/890] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:52 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:01:52.278011 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb099995-13fd-49c8-b2f7-dc0cc9cbb244 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:52.302675 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:52.398089 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb099995-13fd-49c8-b2f7-dc0cc9cbb244 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:52.416679 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452bf1b-df3e-4b22-9574-c4e7946498b1 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:52.416987 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452bf1b-df3e-4b22-9574-c4e7946498b1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:52.417249 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9452bf1b-df3e-4b22-9574-c4e7946498b1 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:52.418946 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 457/891] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:52.425223 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7407cfd6-1a1d-4696-a1a2-eddbc277867b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:52.425612 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7407cfd6-1a1d-4696-a1a2-eddbc277867b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:52.425900 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7407cfd6-1a1d-4696-a1a2-eddbc277867b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:52.434001 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454fc6b5-f94a-4a66-b6da-e9a5e04d5fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:52.434804 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fc6b5-f94a-4a66-b6da-e9a5e04d5fdf None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:52.435111 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fc6b5-f94a-4a66-b6da-e9a5e04d5fdf None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:52.435382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454fc6b5-f94a-4a66-b6da-e9a5e04d5fdf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:52.444438 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7407cfd6-1a1d-4696-a1a2-eddbc277867b None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:52.446316 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7407cfd6-1a1d-4696-a1a2-eddbc277867b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:52.446888 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454fc6b5-f94a-4a66-b6da-e9a5e04d5fdf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:01:52.455561 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-454fc6b5-f94a-4a66-b6da-e9a5e04d5fdf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 15:01:52.455561 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:52.455561 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:52.455561 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:52.455561 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:52.455561 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:52.455561 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:52.455561 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:52.455561 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:52.455561 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:52.455561 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:52.455561 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:52.455561 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:01:52.455561 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:01:52.455561 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:01:52.455561 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:01:52.456529 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:01:52.456529 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:01:52.456529 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:01:52.456529 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:01:52.456529 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:52.456529 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:52.456529 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:52.456529 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:52.456529 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:52.456529 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:52.456529 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:52.456529 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:52.456529 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:52.456529 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:52.456529 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:52.456529 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:52.457387 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:52.457387 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:52.457387 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:52.457387 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:52.457387 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:01:52.457387 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:01:52.457387 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:01:52.457387 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:01:52.457387 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:01:52.457387 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:01:52.457387 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 15:01:52.457387 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:52.457387 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 435/892] 10.222.0.50 () {56 vars in 1120 bytes} [Fri May 27 15:01:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:52.464177 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454fc6b5-f94a-4a66-b6da-e9a5e04d5fdf None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:52.524181 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:52.544779 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 458/893] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:52 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 15:01:52.564140 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7407cfd6-1a1d-4696-a1a2-eddbc277867b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:52.580488 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:52.613956 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454fc6b5-f94a-4a66-b6da-e9a5e04d5fdf None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:52.660144 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b9949-5377-4afd-9e04-f6fa03d38882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:52.661199 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b9949-5377-4afd-9e04-f6fa03d38882 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:52.662222 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b9949-5377-4afd-9e04-f6fa03d38882 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:52.662710 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b9949-5377-4afd-9e04-f6fa03d38882 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:52.664184 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90b9949-5377-4afd-9e04-f6fa03d38882 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:52.664722 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90b9949-5377-4afd-9e04-f6fa03d38882 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:52.667057 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90b9949-5377-4afd-9e04-f6fa03d38882 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:52.683867 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 436/894] 10.222.0.50 () {56 vars in 1115 bytes} [Fri May 27 15:01:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:52.685537 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b9949-5377-4afd-9e04-f6fa03d38882 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:52.696883 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7407cfd6-1a1d-4696-a1a2-eddbc277867b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:52.743716 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a283de0-2568-4f38-af25-af00d202008f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:52.744442 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a283de0-2568-4f38-af25-af00d202008f None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:52.744733 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a283de0-2568-4f38-af25-af00d202008f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:52.745010 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a283de0-2568-4f38-af25-af00d202008f None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:52.746167 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a283de0-2568-4f38-af25-af00d202008f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:52.747827 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a283de0-2568-4f38-af25-af00d202008f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:52.774915 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 459/895] 10.222.0.50 () {60 vars in 1141 bytes} [Fri May 27 15:01:52 2022] POST /identity/v3/services => generated 241 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:01:52.827558 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90b9949-5377-4afd-9e04-f6fa03d38882 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:52.876885 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee2a9c2-85ed-4378-a5c3-df4c80be01f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:52.877801 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee2a9c2-85ed-4378-a5c3-df4c80be01f2 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:52.878092 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee2a9c2-85ed-4378-a5c3-df4c80be01f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:52.878371 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee2a9c2-85ed-4378-a5c3-df4c80be01f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:52.880546 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:52.891876 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee2a9c2-85ed-4378-a5c3-df4c80be01f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:52.893468 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee2a9c2-85ed-4378-a5c3-df4c80be01f2 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:52.912882 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-fee2a9c2-85ed-4378-a5c3-df4c80be01f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:52.912882 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:52.912882 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:52.912882 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:52.912882 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:52.912882 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:52.912882 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:52.912882 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:52.912882 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:52.912882 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:52.912882 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:52.912882 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:52.912882 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:52.912882 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:52.912882 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:52.913852 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:52.913852 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:52.913852 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:52.913852 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:52.913852 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:52.913852 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:52.913852 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:52.913852 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:52.913852 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:52.913852 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:52.913852 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:52.913852 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:52.913852 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:52.913852 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:52.913852 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:52.913852 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:52.913852 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:52.914797 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:52.914797 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:52.914797 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:52.914797 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:52.914797 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 437/896] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:52.922327 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a283de0-2568-4f38-af25-af00d202008f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:52.972252 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf063d-c862-43d4-9232-cf42f1f30ffc None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:52.972590 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf063d-c862-43d4-9232-cf42f1f30ffc None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:52.972819 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf063d-c862-43d4-9232-cf42f1f30ffc None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:52.974210 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 438/897] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:52.979346 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb490a02-fc13-4487-867d-e4c983c0b3c8 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:52.979638 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb490a02-fc13-4487-867d-e4c983c0b3c8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:52.979926 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb490a02-fc13-4487-867d-e4c983c0b3c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:52.981224 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:53.000380 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-cb490a02-fc13-4487-867d-e4c983c0b3c8 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:53.001825 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cb490a02-fc13-4487-867d-e4c983c0b3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:53.042356 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a283de0-2568-4f38-af25-af00d202008f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:53.085841 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 439/898] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 15:01:53.086063 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd226f13-bb9d-4d0f-8407-09bcccfa098b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:53.086063 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd226f13-bb9d-4d0f-8407-09bcccfa098b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:53.086063 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd226f13-bb9d-4d0f-8407-09bcccfa098b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:53.086063 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd226f13-bb9d-4d0f-8407-09bcccfa098b None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:53.086349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd226f13-bb9d-4d0f-8407-09bcccfa098b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:53.086349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd226f13-bb9d-4d0f-8407-09bcccfa098b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:53.087393 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd226f13-bb9d-4d0f-8407-09bcccfa098b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:53.101336 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 460/899] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:01:53.105688 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb490a02-fc13-4487-867d-e4c983c0b3c8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:53.110588 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd226f13-bb9d-4d0f-8407-09bcccfa098b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:53.228069 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb490a02-fc13-4487-867d-e4c983c0b3c8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:53.246790 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:53.249293 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd226f13-bb9d-4d0f-8407-09bcccfa098b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:53.262880 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:53.270254 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c431991-61a3-4d8a-b251-e6ac48a5e9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:53.271479 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c431991-61a3-4d8a-b251-e6ac48a5e9f2 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:53.271990 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c431991-61a3-4d8a-b251-e6ac48a5e9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:53.272471 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c431991-61a3-4d8a-b251-e6ac48a5e9f2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:53.273346 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c431991-61a3-4d8a-b251-e6ac48a5e9f2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:53.275027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c431991-61a3-4d8a-b251-e6ac48a5e9f2 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:53.288511 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aee50c-f6d5-4297-9237-07ea9aaf2db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:53.289326 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aee50c-f6d5-4297-9237-07ea9aaf2db4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:53.289652 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-6c431991-61a3-4d8a-b251-e6ac48a5e9f2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 15:01:53.289652 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:53.289652 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 15:01:53.289652 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 15:01:53.289652 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 15:01:53.289652 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 15:01:53.289652 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 15:01:53.289652 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:01:53.289652 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 15:01:53.289652 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:53.289652 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 15:01:53.289652 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:53.289652 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:53.289652 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:53.289652 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:53.289652 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:53.290581 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:53.290581 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:53.290581 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:53.290581 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:53.290581 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:53.290581 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:53.290581 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:53.290581 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 15:01:53.290581 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 15:01:53.290581 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 15:01:53.290581 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 15:01:53.290581 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:53.290581 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:53.290581 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:53.290581 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:53.290581 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:53.291463 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:53.291463 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:53.291463 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:53.291463 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:53.291463 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:53.291463 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:53.291463 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:53.291463 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:53.291463 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:53.291463 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:53.291463 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:53.291463 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 15:01:53.291463 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:01:53.291463 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 15:01:53.291463 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:53.292345 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aee50c-f6d5-4297-9237-07ea9aaf2db4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:53.292345 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aee50c-f6d5-4297-9237-07ea9aaf2db4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:53.298895 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aee50c-f6d5-4297-9237-07ea9aaf2db4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:53.299256 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 440/900] 10.222.0.50 () {56 vars in 1132 bytes} [Fri May 27 15:01:53 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:53.299386 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aee50c-f6d5-4297-9237-07ea9aaf2db4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:53.300699 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c431991-61a3-4d8a-b251-e6ac48a5e9f2 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:53.315647 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-e0aee50c-f6d5-4297-9237-07ea9aaf2db4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 15:01:53.315647 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:53.315647 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:53.315647 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:53.315647 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:53.315647 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:53.315647 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:53.315647 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:53.315647 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:53.315647 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:53.315647 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:53.315647 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:53.315647 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 15:01:53.315647 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 15:01:53.315647 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 15:01:53.315647 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 15:01:53.316697 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:53.316697 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:53.316697 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 15:01:53.316697 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:53.316697 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 15:01:53.316697 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:53.316697 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:53.316697 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:53.316697 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:53.316697 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:53.316697 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:53.316697 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:53.316697 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:53.316697 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:53.316697 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:53.316697 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:53.316697 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:53.316697 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:53.317652 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:53.317652 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:53.317652 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 15:01:53.317652 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 15:01:53.317652 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 15:01:53.317652 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 15:01:53.317652 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 15:01:53.317652 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 15:01:53.317652 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 15:01:53.317652 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:53.317652 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 461/901] 10.222.0.50 () {56 vars in 1117 bytes} [Fri May 27 15:01:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:01:53.322780 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aee50c-f6d5-4297-9237-07ea9aaf2db4 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:53.427531 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c431991-61a3-4d8a-b251-e6ac48a5e9f2 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:53.445892 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aee50c-f6d5-4297-9237-07ea9aaf2db4 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:53.465029 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc5c205-7556-4cb7-ae37-2c6b1eb8743c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:53.465829 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc5c205-7556-4cb7-ae37-2c6b1eb8743c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:53.466209 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc5c205-7556-4cb7-ae37-2c6b1eb8743c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:53.466581 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc5c205-7556-4cb7-ae37-2c6b1eb8743c None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:53.467653 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc5c205-7556-4cb7-ae37-2c6b1eb8743c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:53.468092 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc5c205-7556-4cb7-ae37-2c6b1eb8743c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:53.469551 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc5c205-7556-4cb7-ae37-2c6b1eb8743c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:53.475323 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 441/902] 10.222.0.50 () {56 vars in 1126 bytes} [Fri May 27 15:01:53 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:53.480654 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc5c205-7556-4cb7-ae37-2c6b1eb8743c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:53.488047 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a1c518-3221-4c23-9b8d-bd287dff6fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:53.488762 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1c518-3221-4c23-9b8d-bd287dff6fc6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:53.489049 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1c518-3221-4c23-9b8d-bd287dff6fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:53.489380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1c518-3221-4c23-9b8d-bd287dff6fc6 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:53.490226 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a1c518-3221-4c23-9b8d-bd287dff6fc6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:53.490537 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a1c518-3221-4c23-9b8d-bd287dff6fc6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:53.492003 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a1c518-3221-4c23-9b8d-bd287dff6fc6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:53.511221 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 462/903] 10.222.0.50 () {56 vars in 1149 bytes} [Fri May 27 15:01:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:53.520749 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a1c518-3221-4c23-9b8d-bd287dff6fc6 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:53.541019 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:53.583318 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:53.613821 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc5c205-7556-4cb7-ae37-2c6b1eb8743c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:53.630410 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a1c518-3221-4c23-9b8d-bd287dff6fc6 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:53.651563 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d40480-f207-45b7-a934-70e107ece8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:53.652231 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d40480-f207-45b7-a934-70e107ece8e1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:53.652514 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d40480-f207-45b7-a934-70e107ece8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:53.652791 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d40480-f207-45b7-a934-70e107ece8e1 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:53.653496 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d40480-f207-45b7-a934-70e107ece8e1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:53.653803 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d40480-f207-45b7-a934-70e107ece8e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:53.656031 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d40480-f207-45b7-a934-70e107ece8e1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:53.666451 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 442/904] 10.222.0.50 () {56 vars in 1126 bytes} [Fri May 27 15:01:53 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:53.667770 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d40480-f207-45b7-a934-70e107ece8e1 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:53.677225 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e653b0ac-7ba7-4b43-bd48-101e5cdb439c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:53.677884 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e653b0ac-7ba7-4b43-bd48-101e5cdb439c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:53.678169 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e653b0ac-7ba7-4b43-bd48-101e5cdb439c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:53.678444 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e653b0ac-7ba7-4b43-bd48-101e5cdb439c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:53.772324 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e653b0ac-7ba7-4b43-bd48-101e5cdb439c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:53.774264 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e653b0ac-7ba7-4b43-bd48-101e5cdb439c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:53.792795 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d40480-f207-45b7-a934-70e107ece8e1 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:53.803733 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:53.812984 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:53.840349 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead1b133-e39a-4613-8bfb-049484b3e590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:53.841017 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead1b133-e39a-4613-8bfb-049484b3e590 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:53.841303 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead1b133-e39a-4613-8bfb-049484b3e590 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:53.841581 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead1b133-e39a-4613-8bfb-049484b3e590 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:53.843255 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead1b133-e39a-4613-8bfb-049484b3e590 None admin] RBAC: Adding query filter params (interface=public, service_id=c87e9016745f416683fb15fe3b4764e0, region_id=RegionOne) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:53.843575 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead1b133-e39a-4613-8bfb-049484b3e590 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:53.845531 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead1b133-e39a-4613-8bfb-049484b3e590 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:53.857180 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 443/905] 10.222.0.50 () {56 vars in 1251 bytes} [Fri May 27 15:01:53 2022] GET /identity/v3/endpoints?service_id=c87e9016745f416683fb15fe3b4764e0&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:53.859389 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead1b133-e39a-4613-8bfb-049484b3e590 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:53.900844 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 463/906] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:01:53 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 27 15:01:53.908215 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e653b0ac-7ba7-4b43-bd48-101e5cdb439c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:53.972093 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead1b133-e39a-4613-8bfb-049484b3e590 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:54.008658 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce3827f-55e2-4fca-87fc-88d4f6940cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:54.009330 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3827f-55e2-4fca-87fc-88d4f6940cc7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:54.009613 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3827f-55e2-4fca-87fc-88d4f6940cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:54.009891 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3827f-55e2-4fca-87fc-88d4f6940cc7 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:54.010623 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce3827f-55e2-4fca-87fc-88d4f6940cc7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:54.011959 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce3827f-55e2-4fca-87fc-88d4f6940cc7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:54.018299 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e653b0ac-7ba7-4b43-bd48-101e5cdb439c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:54.026969 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 444/907] 10.222.0.50 () {56 vars in 1087 bytes} [Fri May 27 15:01:53 2022] GET /identity/v3/services => generated 1497 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:01:54.062342 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:54.071906 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555bedbd-c4f7-491f-81b6-21df3dad4c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:54.072734 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555bedbd-c4f7-491f-81b6-21df3dad4c46 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:54.073019 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555bedbd-c4f7-491f-81b6-21df3dad4c46 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:54.073299 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555bedbd-c4f7-491f-81b6-21df3dad4c46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:54.086094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555bedbd-c4f7-491f-81b6-21df3dad4c46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:54.087621 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555bedbd-c4f7-491f-81b6-21df3dad4c46 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:54.112094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-555bedbd-c4f7-491f-81b6-21df3dad4c46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:54.112094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:54.112094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:54.112094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:54.112094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:54.112094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:54.112094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:54.112094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:54.112094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:54.112094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:54.112094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:54.112094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:54.112094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:54.112094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:54.112094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:54.113094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:54.113094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:54.113094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:54.113094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:54.113094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:54.113094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:54.113094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:54.113094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:54.113094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:54.113094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:54.113094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:54.113094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:54.113094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:54.113094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:54.113094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:54.113094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:54.113094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:54.114424 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:54.114424 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:54.114424 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:54.114424 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:54.114424 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 464/908] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:01:54.119509 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce3827f-55e2-4fca-87fc-88d4f6940cc7 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:54.165614 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:54.244177 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce3827f-55e2-4fca-87fc-88d4f6940cc7 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:54.244950 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35509524-1bcb-4ab8-a42a-1e48531d3371 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:54.245238 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35509524-1bcb-4ab8-a42a-1e48531d3371 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:54.245515 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35509524-1bcb-4ab8-a42a-1e48531d3371 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:54.246903 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 465/909] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 15:01:54.252901 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3762c3-384c-4417-9adc-8748073fc05f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:54.253185 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3762c3-384c-4417-9adc-8748073fc05f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:54.253465 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3762c3-384c-4417-9adc-8748073fc05f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:54.273778 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6e3762c3-384c-4417-9adc-8748073fc05f None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:54.275745 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6e3762c3-384c-4417-9adc-8748073fc05f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:54.284838 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4095d0-21cb-4a8e-be49-1e574ea446ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:54.285538 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4095d0-21cb-4a8e-be49-1e574ea446ca None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:54.285834 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4095d0-21cb-4a8e-be49-1e574ea446ca None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:54.286115 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4095d0-21cb-4a8e-be49-1e574ea446ca None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:54.286833 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4095d0-21cb-4a8e-be49-1e574ea446ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:54.287171 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4095d0-21cb-4a8e-be49-1e574ea446ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:54.288538 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4095d0-21cb-4a8e-be49-1e574ea446ca None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:54.301078 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 445/910] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:54.306969 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4095d0-21cb-4a8e-be49-1e574ea446ca None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:54.358871 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 466/911] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 15:01:54.377871 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3762c3-384c-4417-9adc-8748073fc05f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:54.411286 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:54.419243 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4095d0-21cb-4a8e-be49-1e574ea446ca None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:54.464368 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae9ebc-017a-46f5-b5b9-0febe1838147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:54.465171 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae9ebc-017a-46f5-b5b9-0febe1838147 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:54.465458 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae9ebc-017a-46f5-b5b9-0febe1838147 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:54.465737 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae9ebc-017a-46f5-b5b9-0febe1838147 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:54.474381 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffae9ebc-017a-46f5-b5b9-0febe1838147 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:54.476392 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffae9ebc-017a-46f5-b5b9-0febe1838147 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:54.494046 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-ffae9ebc-017a-46f5-b5b9-0febe1838147 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:54.494046 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:54.494046 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:54.494046 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:54.494046 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:54.494046 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:54.494046 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:54.494046 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:54.494046 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:54.494046 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:54.494046 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:54.494046 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:54.494046 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 15:01:54.494046 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 15:01:54.494046 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 15:01:54.495120 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 15:01:54.495120 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:54.495120 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:54.495120 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:54.495120 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:54.495120 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:54.495120 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:54.495120 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:54.495120 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:54.495120 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:54.495120 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:54.495120 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:54.495120 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:54.495120 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:54.495120 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:54.495120 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:54.495120 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:54.496027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 15:01:54.496027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 15:01:54.496027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 15:01:54.496027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 15:01:54.496027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 15:01:54.496027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 15:01:54.496027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:54.496027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:54.496027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 446/912] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:01:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:54.503367 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae9ebc-017a-46f5-b5b9-0febe1838147 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:54.515694 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3762c3-384c-4417-9adc-8748073fc05f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:54.531227 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:54.563107 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0403b-0aa9-4987-ae80-42ab706c3b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:54.564331 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0403b-0aa9-4987-ae80-42ab706c3b6a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:54.564685 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0403b-0aa9-4987-ae80-42ab706c3b6a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:54.565303 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb0403b-0aa9-4987-ae80-42ab706c3b6a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:54.566045 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb0403b-0aa9-4987-ae80-42ab706c3b6a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:54.567803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb0403b-0aa9-4987-ae80-42ab706c3b6a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:54.584312 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-6fb0403b-0aa9-4987-ae80-42ab706c3b6a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 15:01:54.584312 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:54.584312 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 15:01:54.584312 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 15:01:54.584312 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 15:01:54.584312 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 15:01:54.584312 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 15:01:54.584312 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:01:54.584312 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 15:01:54.584312 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:54.584312 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 15:01:54.584312 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:54.584312 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:54.584312 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:54.584312 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:54.584312 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:54.584312 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:54.585423 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:54.585423 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:54.585423 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:54.585423 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:54.585423 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:54.585423 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:54.585423 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 15:01:54.585423 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 15:01:54.585423 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 15:01:54.585423 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 15:01:54.585423 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:54.585423 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:54.585423 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:54.585423 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:54.585423 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:54.585423 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:54.586290 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:54.586290 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:54.586290 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:54.586290 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:54.586290 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:54.586290 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:54.586290 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:54.586290 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:54.586290 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:54.586290 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:54.586290 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 15:01:54.586290 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:01:54.586290 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 27 15:01:54.586290 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:54.586290 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 467/913] 10.222.0.50 () {56 vars in 1114 bytes} [Fri May 27 15:01:54 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:01:54.592101 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0403b-0aa9-4987-ae80-42ab706c3b6a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:54.643810 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae9ebc-017a-46f5-b5b9-0febe1838147 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:54.690305 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfb07d3-eae3-4181-8364-5157775c304e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:54.691397 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb07d3-eae3-4181-8364-5157775c304e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:54.691891 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb07d3-eae3-4181-8364-5157775c304e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:54.692369 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb07d3-eae3-4181-8364-5157775c304e None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:54.693937 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfb07d3-eae3-4181-8364-5157775c304e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:54.694478 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfb07d3-eae3-4181-8364-5157775c304e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:54.696690 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfb07d3-eae3-4181-8364-5157775c304e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:54.719010 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 447/914] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 15:01:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:01:54.720440 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfb07d3-eae3-4181-8364-5157775c304e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:54.729235 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb0403b-0aa9-4987-ae80-42ab706c3b6a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:54.739762 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:54.769575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd256954-cb5a-4db1-8717-92b997b105f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:54.770662 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd256954-cb5a-4db1-8717-92b997b105f6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:54.771236 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd256954-cb5a-4db1-8717-92b997b105f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:54.771751 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd256954-cb5a-4db1-8717-92b997b105f6 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:54.772847 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd256954-cb5a-4db1-8717-92b997b105f6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:54.773368 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd256954-cb5a-4db1-8717-92b997b105f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:54.774986 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd256954-cb5a-4db1-8717-92b997b105f6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:54.780905 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 468/915] 10.222.0.50 () {56 vars in 1114 bytes} [Fri May 27 15:01:54 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:54.786982 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd256954-cb5a-4db1-8717-92b997b105f6 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:54.824310 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfb07d3-eae3-4181-8364-5157775c304e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:54.832574 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:54.860552 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a702e80e-db0f-4685-b600-b363bafd523d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:54.861222 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702e80e-db0f-4685-b600-b363bafd523d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:54.861947 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702e80e-db0f-4685-b600-b363bafd523d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:54.861947 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702e80e-db0f-4685-b600-b363bafd523d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:54.862617 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a702e80e-db0f-4685-b600-b363bafd523d None admin] RBAC: Adding query filter params (role.id=4564db14e2a543f18cb88a5bea5aada2, scope.project.id=daf12585428b41a2b45bdbc85413a6c4, user.id=09c23d1c732c452a9597c647ad0241f0) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:54.862945 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a702e80e-db0f-4685-b600-b363bafd523d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:54.864403 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a702e80e-db0f-4685-b600-b363bafd523d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:54.870676 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 448/916] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:01:54 2022] GET /identity/v3/role_assignments?user.id=09c23d1c732c452a9597c647ad0241f0&scope.project.id=daf12585428b41a2b45bdbc85413a6c4&role.id=4564db14e2a543f18cb88a5bea5aada2 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:54.909814 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd256954-cb5a-4db1-8717-92b997b105f6 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:54.954997 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56354739-0f92-4b90-b1a8-e924220018cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:54.956006 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56354739-0f92-4b90-b1a8-e924220018cb None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:54.956442 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56354739-0f92-4b90-b1a8-e924220018cb None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:54.956869 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56354739-0f92-4b90-b1a8-e924220018cb None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:54.957783 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56354739-0f92-4b90-b1a8-e924220018cb None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:54.958253 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56354739-0f92-4b90-b1a8-e924220018cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:54.960036 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56354739-0f92-4b90-b1a8-e924220018cb None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:54.970841 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 469/917] 10.222.0.50 () {56 vars in 1114 bytes} [Fri May 27 15:01:54 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:54.976386 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a702e80e-db0f-4685-b600-b363bafd523d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:55.069587 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:55.091213 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a702e80e-db0f-4685-b600-b363bafd523d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:55.126582 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af6a8a8-33ad-4fdb-8b22-517c351ae913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:55.127351 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af6a8a8-33ad-4fdb-8b22-517c351ae913 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:55.127772 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af6a8a8-33ad-4fdb-8b22-517c351ae913 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:55.128051 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af6a8a8-33ad-4fdb-8b22-517c351ae913 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:55.128803 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af6a8a8-33ad-4fdb-8b22-517c351ae913 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:55.130177 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af6a8a8-33ad-4fdb-8b22-517c351ae913 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:55.141689 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf3360-48d3-4ea3-96b3-e800e098a964 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:55.142010 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf3360-48d3-4ea3-96b3-e800e098a964 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:55.142234 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf3360-48d3-4ea3-96b3-e800e098a964 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:55.143621 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 470/918] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 15:01:55.151069 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eaedaa-b40c-4291-87a1-5b1b5dc4b961 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:55.151652 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eaedaa-b40c-4291-87a1-5b1b5dc4b961 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:55.152370 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eaedaa-b40c-4291-87a1-5b1b5dc4b961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:55.155204 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 449/919] 10.222.0.50 () {60 vars in 1144 bytes} [Fri May 27 15:01:54 2022] POST /identity/v3/endpoints => generated 346 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:01:55.177411 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a0eaedaa-b40c-4291-87a1-5b1b5dc4b961 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:55.178640 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a0eaedaa-b40c-4291-87a1-5b1b5dc4b961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:55.264346 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 471/920] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 15:01:55.287388 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af6a8a8-33ad-4fdb-8b22-517c351ae913 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:55.403115 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af6a8a8-33ad-4fdb-8b22-517c351ae913 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:55.439929 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47725dfd-4818-47f4-87e1-a1f3984a229b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:55.440743 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47725dfd-4818-47f4-87e1-a1f3984a229b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:55.441075 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47725dfd-4818-47f4-87e1-a1f3984a229b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:55.441350 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47725dfd-4818-47f4-87e1-a1f3984a229b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:55.442000 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47725dfd-4818-47f4-87e1-a1f3984a229b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:55.443404 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47725dfd-4818-47f4-87e1-a1f3984a229b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:55.446664 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:55.457892 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-47725dfd-4818-47f4-87e1-a1f3984a229b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 15:01:55.457892 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:55.457892 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 15:01:55.457892 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 15:01:55.457892 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 15:01:55.457892 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 15:01:55.457892 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 15:01:55.457892 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:01:55.457892 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 15:01:55.457892 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:55.457892 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 15:01:55.457892 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:55.457892 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:55.457892 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:55.457892 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:55.457892 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:55.457892 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:55.459041 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:55.459041 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:55.459041 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:55.459041 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:55.459041 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:55.459041 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:55.459041 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 15:01:55.459041 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 15:01:55.459041 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 15:01:55.459041 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 15:01:55.459041 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:55.459041 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:55.459041 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:55.459041 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:55.459041 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:55.459041 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:55.459887 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:55.459887 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:55.459887 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:55.459887 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:55.459887 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:55.459887 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:55.459887 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:55.459887 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:55.459887 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:55.459887 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:55.459887 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 15:01:55.459887 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:01:55.459887 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 15:01:55.459887 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:55.459887 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 450/921] 10.222.0.50 () {56 vars in 1126 bytes} [Fri May 27 15:01:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:55.467748 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eaedaa-b40c-4291-87a1-5b1b5dc4b961 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:55.498349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:55.571886 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eaedaa-b40c-4291-87a1-5b1b5dc4b961 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:55.615023 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d56bfb4-7dcc-4d76-b160-d8d258b5d339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:55.615691 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d56bfb4-7dcc-4d76-b160-d8d258b5d339 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:55.615973 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d56bfb4-7dcc-4d76-b160-d8d258b5d339 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:55.616248 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d56bfb4-7dcc-4d76-b160-d8d258b5d339 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:55.616951 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d56bfb4-7dcc-4d76-b160-d8d258b5d339 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:55.617257 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d56bfb4-7dcc-4d76-b160-d8d258b5d339 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:55.618623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d56bfb4-7dcc-4d76-b160-d8d258b5d339 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:55.624140 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 472/922] 10.222.0.50 () {56 vars in 1122 bytes} [Fri May 27 15:01:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:55.629492 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47725dfd-4818-47f4-87e1-a1f3984a229b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:55.735693 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47725dfd-4818-47f4-87e1-a1f3984a229b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:55.772374 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33da6f5-82bc-47e8-9f15-3bacf0d215f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:55.773043 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33da6f5-82bc-47e8-9f15-3bacf0d215f2 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:55.773376 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33da6f5-82bc-47e8-9f15-3bacf0d215f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:55.773658 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33da6f5-82bc-47e8-9f15-3bacf0d215f2 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:55.774365 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33da6f5-82bc-47e8-9f15-3bacf0d215f2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:55.774677 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33da6f5-82bc-47e8-9f15-3bacf0d215f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:55.776094 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33da6f5-82bc-47e8-9f15-3bacf0d215f2 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:55.782322 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 451/923] 10.222.0.50 () {56 vars in 1122 bytes} [Fri May 27 15:01:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:55.788164 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d56bfb4-7dcc-4d76-b160-d8d258b5d339 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:55.881132 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:55.891370 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d56bfb4-7dcc-4d76-b160-d8d258b5d339 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:55.929896 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af2af64-0cf9-484f-9bb1-a66a66a40144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:55.930556 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af2af64-0cf9-484f-9bb1-a66a66a40144 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:55.930862 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af2af64-0cf9-484f-9bb1-a66a66a40144 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:55.931146 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af2af64-0cf9-484f-9bb1-a66a66a40144 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:55.931945 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af2af64-0cf9-484f-9bb1-a66a66a40144 None admin] RBAC: Adding query filter params (interface=public, service_id=3ff7d3e34aa04d77b50dfc3d6e932c4f, region_id=RegionOne) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:55.932492 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af2af64-0cf9-484f-9bb1-a66a66a40144 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:55.933931 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af2af64-0cf9-484f-9bb1-a66a66a40144 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:55.942944 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 473/924] 10.222.0.50 () {56 vars in 1251 bytes} [Fri May 27 15:01:55 2022] GET /identity/v3/endpoints?service_id=3ff7d3e34aa04d77b50dfc3d6e932c4f&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:55.948391 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33da6f5-82bc-47e8-9f15-3bacf0d215f2 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:56.016397 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:56.062202 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33da6f5-82bc-47e8-9f15-3bacf0d215f2 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:56.099399 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eb17f5-43ba-43e3-a1ff-03c71b6f339b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:56.100070 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb17f5-43ba-43e3-a1ff-03c71b6f339b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:56.100358 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb17f5-43ba-43e3-a1ff-03c71b6f339b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:56.100635 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb17f5-43ba-43e3-a1ff-03c71b6f339b None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:56.101327 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eb17f5-43ba-43e3-a1ff-03c71b6f339b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:56.102702 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eb17f5-43ba-43e3-a1ff-03c71b6f339b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:56.109591 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 452/925] 10.222.0.50 () {56 vars in 1087 bytes} [Fri May 27 15:01:55 2022] GET /identity/v3/services => generated 1497 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:01:56.502715 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9dd82-0875-4d72-aba3-694f682839d8 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:56.503398 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9dd82-0875-4d72-aba3-694f682839d8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:56.503878 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da9dd82-0875-4d72-aba3-694f682839d8 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:56.505712 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 474/926] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:56.511068 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030f001-9eb2-4b91-a412-472a29a8e4d8 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:56.511367 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030f001-9eb2-4b91-a412-472a29a8e4d8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:56.511652 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030f001-9eb2-4b91-a412-472a29a8e4d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:56.529952 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3030f001-9eb2-4b91-a412-472a29a8e4d8 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:56.531178 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3030f001-9eb2-4b91-a412-472a29a8e4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:56.603645 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 453/927] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:01:56.626479 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9dd82-0875-4d72-aba3-694f682839d8 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:56.742902 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da9dd82-0875-4d72-aba3-694f682839d8 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:56.778666 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f834f4-5aec-41cf-88ca-2e12767b8592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:56.779587 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f834f4-5aec-41cf-88ca-2e12767b8592 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:56.779874 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f834f4-5aec-41cf-88ca-2e12767b8592 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:56.780151 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f834f4-5aec-41cf-88ca-2e12767b8592 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:56.780862 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f834f4-5aec-41cf-88ca-2e12767b8592 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:56.782267 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f834f4-5aec-41cf-88ca-2e12767b8592 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:56.795456 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-96f834f4-5aec-41cf-88ca-2e12767b8592 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 15:01:56.795456 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:56.795456 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 15:01:56.795456 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 15:01:56.795456 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 15:01:56.795456 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 15:01:56.795456 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 15:01:56.795456 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:01:56.795456 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 15:01:56.795456 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:56.795456 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 15:01:56.795456 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:56.795456 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:56.795456 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:56.795456 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:56.795456 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:56.796409 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:56.796409 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:56.796409 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:56.796409 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:56.796409 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:56.796409 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:56.796409 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:56.796409 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 15:01:56.796409 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 15:01:56.796409 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 15:01:56.796409 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 15:01:56.796409 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:56.796409 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:56.796409 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:56.796409 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:56.796409 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:56.797242 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:56.797242 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:56.797242 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:56.797242 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:56.797242 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:56.797242 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:56.797242 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:56.797242 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:56.797242 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:56.797242 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:56.797242 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:56.797242 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 15:01:56.797242 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:01:56.797242 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 27 15:01:56.797242 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:56.797242 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 475/928] 10.222.0.50 () {56 vars in 1132 bytes} [Fri May 27 15:01:56 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:56.804472 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3030f001-9eb2-4b91-a412-472a29a8e4d8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:56.867513 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:56.910258 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3030f001-9eb2-4b91-a412-472a29a8e4d8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:56.946453 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02563cea-0ea9-464b-a0c4-acce6726b1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:56.947488 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02563cea-0ea9-464b-a0c4-acce6726b1e8 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:56.947971 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02563cea-0ea9-464b-a0c4-acce6726b1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:56.948463 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02563cea-0ea9-464b-a0c4-acce6726b1e8 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:56.949373 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02563cea-0ea9-464b-a0c4-acce6726b1e8 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:56.949890 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02563cea-0ea9-464b-a0c4-acce6726b1e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:56.956821 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02563cea-0ea9-464b-a0c4-acce6726b1e8 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:56.962514 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 454/929] 10.222.0.50 () {56 vars in 1126 bytes} [Fri May 27 15:01:56 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:56.968915 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f834f4-5aec-41cf-88ca-2e12767b8592 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:57.005345 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:57.097622 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f834f4-5aec-41cf-88ca-2e12767b8592 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:57.132626 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1947a1ee-8deb-40df-a3b2-90cd1d880ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:57.133295 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947a1ee-8deb-40df-a3b2-90cd1d880ecf None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:57.133619 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947a1ee-8deb-40df-a3b2-90cd1d880ecf None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:57.133867 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947a1ee-8deb-40df-a3b2-90cd1d880ecf None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:57.134538 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1947a1ee-8deb-40df-a3b2-90cd1d880ecf None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:57.134869 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1947a1ee-8deb-40df-a3b2-90cd1d880ecf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:57.136236 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1947a1ee-8deb-40df-a3b2-90cd1d880ecf None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:57.141937 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 476/930] 10.222.0.50 () {56 vars in 1126 bytes} [Fri May 27 15:01:56 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:57.147095 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02563cea-0ea9-464b-a0c4-acce6726b1e8 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:57.237408 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b07bc-0c52-443f-a257-80ea78293ad4 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:57.237758 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b07bc-0c52-443f-a257-80ea78293ad4 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:57.237978 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b07bc-0c52-443f-a257-80ea78293ad4 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:57.239658 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 477/931] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 15:01:57.244626 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1f485-2cd2-4fae-a612-047727ed458f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:57.244922 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1f485-2cd2-4fae-a612-047727ed458f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:57.245202 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a1f485-2cd2-4fae-a612-047727ed458f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:57.257726 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02563cea-0ea9-464b-a0c4-acce6726b1e8 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:57.262158 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-76a1f485-2cd2-4fae-a612-047727ed458f None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:57.263365 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-76a1f485-2cd2-4fae-a612-047727ed458f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:57.285466 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:57.304719 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ffd32-d838-4f13-877b-8e2d80518d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:57.305386 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ffd32-d838-4f13-877b-8e2d80518d97 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:57.305675 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ffd32-d838-4f13-877b-8e2d80518d97 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:57.305955 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ffd32-d838-4f13-877b-8e2d80518d97 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:57.306714 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878ffd32-d838-4f13-877b-8e2d80518d97 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:57.308676 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878ffd32-d838-4f13-877b-8e2d80518d97 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:57.339146 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 455/933] 10.222.0.50 () {60 vars in 1144 bytes} [Fri May 27 15:01:57 2022] POST /identity/v3/endpoints => generated 347 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:01:57.340974 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 478/933] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:01:57.363097 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ffd32-d838-4f13-877b-8e2d80518d97 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:57.492374 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ffd32-d838-4f13-877b-8e2d80518d97 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:57.532506 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea02c2d-a7e5-4f0d-91f9-80c1d5c786c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:57.533670 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea02c2d-a7e5-4f0d-91f9-80c1d5c786c8 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:57.534168 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea02c2d-a7e5-4f0d-91f9-80c1d5c786c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:57.534629 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea02c2d-a7e5-4f0d-91f9-80c1d5c786c8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:57.546693 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea02c2d-a7e5-4f0d-91f9-80c1d5c786c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:01:57.551731 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef649f6-2d09-448c-ad8b-9456a5420cd8 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:57.552027 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef649f6-2d09-448c-ad8b-9456a5420cd8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:57.552304 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef649f6-2d09-448c-ad8b-9456a5420cd8 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:57.553790 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 479/934] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:57.555014 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-9ea02c2d-a7e5-4f0d-91f9-80c1d5c786c8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 15:01:57.555014 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:57.555014 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:57.555014 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:57.555014 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:57.555014 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:57.555014 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:57.555014 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:57.555014 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:57.555014 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:57.555014 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:57.555014 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:57.555014 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:01:57.555014 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:01:57.555014 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:01:57.555014 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:01:57.555937 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:01:57.555937 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:01:57.555937 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:01:57.555937 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:01:57.555937 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:57.555937 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:57.555937 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:57.555937 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:57.555937 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:57.555937 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:57.555937 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:57.555937 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:57.555937 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:57.555937 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:57.555937 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:57.555937 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:57.556796 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:57.556796 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:57.556796 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:57.556796 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:57.556796 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:01:57.556796 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:01:57.556796 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:01:57.556796 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:01:57.556796 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:01:57.556796 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:01:57.556796 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 15:01:57.556796 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:57.556796 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 456/935] 10.222.0.50 () {56 vars in 1120 bytes} [Fri May 27 15:01:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:57.562194 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea02c2d-a7e5-4f0d-91f9-80c1d5c786c8 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:57.562431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21a4c2-a72a-4424-b438-3261d12651e5 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:57.562725 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21a4c2-a72a-4424-b438-3261d12651e5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:57.563041 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21a4c2-a72a-4424-b438-3261d12651e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:57.586370 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-da21a4c2-a72a-4424-b438-3261d12651e5 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:57.587664 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-da21a4c2-a72a-4424-b438-3261d12651e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:57.599881 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:57.606079 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:57.685223 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 480/936] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:01:57.699611 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea02c2d-a7e5-4f0d-91f9-80c1d5c786c8 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:57.709705 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da21a4c2-a72a-4424-b438-3261d12651e5 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:57.745926 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872951b0-3d7c-493f-9cce-e2e360fbcb74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:57.747188 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872951b0-3d7c-493f-9cce-e2e360fbcb74 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:57.747910 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872951b0-3d7c-493f-9cce-e2e360fbcb74 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:57.748928 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872951b0-3d7c-493f-9cce-e2e360fbcb74 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:57.750167 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872951b0-3d7c-493f-9cce-e2e360fbcb74 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:57.750683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872951b0-3d7c-493f-9cce-e2e360fbcb74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:57.753491 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872951b0-3d7c-493f-9cce-e2e360fbcb74 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:57.765080 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 457/937] 10.222.0.50 () {56 vars in 1115 bytes} [Fri May 27 15:01:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:01:57.770368 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872951b0-3d7c-493f-9cce-e2e360fbcb74 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:57.829014 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da21a4c2-a72a-4424-b438-3261d12651e5 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:57.868083 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8682c8f2-9fa2-4293-accb-ac09f40a9b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:57.868836 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682c8f2-9fa2-4293-accb-ac09f40a9b57 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:57.869116 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682c8f2-9fa2-4293-accb-ac09f40a9b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:57.869395 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682c8f2-9fa2-4293-accb-ac09f40a9b57 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:57.870338 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8682c8f2-9fa2-4293-accb-ac09f40a9b57 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:57.871642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8682c8f2-9fa2-4293-accb-ac09f40a9b57 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:57.873055 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: WARNING keystone.server.flask.common [None req-8682c8f2-9fa2-4293-accb-ac09f40a9b57 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 27 15:01:57.889109 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:57.894632 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.sql.core [None req-8682c8f2-9fa2-4293-accb-ac09f40a9b57 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 27 15:01:57.894632 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 27 15:01:57.894632 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [parameters: {'id': '5f62ad2d08cb408aae9d2271812d9c0f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 27 15:01:57.894632 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80395) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 27 15:01:57.900433 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:57.905953 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-8682c8f2-9fa2-4293-accb-ac09f40a9b57 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 5f62ad2d08cb408aae9d2271812d9c0f.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 5f62ad2d08cb408aae9d2271812d9c0f. May 27 15:01:57.905953 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:57.905953 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 15:01:57.905953 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 15:01:57.905953 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 15:01:57.905953 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 15:01:57.905953 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 15:01:57.905953 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result = self._query(query) May 27 15:01:57.905953 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 15:01:57.905953 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application conn.query(q) May 27 15:01:57.905953 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 15:01:57.905953 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 15:01:57.905953 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 15:01:57.906792 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result.read() May 27 15:01:57.906792 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 15:01:57.906792 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 15:01:57.906792 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 15:01:57.906792 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 15:01:57.906792 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 15:01:57.906792 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 15:01:57.906792 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 15:01:57.906792 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 15:01:57.906792 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 27 15:01:57.906792 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:57.906792 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 27 15:01:57.906792 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:57.906792 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:57.906792 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 27 15:01:57.906792 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 27 15:01:57.906792 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 27 15:01:57.906792 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return project_ref.to_dict() May 27 15:01:57.907811 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 15:01:57.907811 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application next(self.gen) May 27 15:01:57.907811 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 27 15:01:57.907811 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application yield resource May 27 15:01:57.907811 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 15:01:57.907811 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application next(self.gen) May 27 15:01:57.907811 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 27 15:01:57.907811 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.session.rollback() May 27 15:01:57.907811 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 27 15:01:57.907811 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.force_reraise() May 27 15:01:57.907811 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 27 15:01:57.907811 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise self.value May 27 15:01:57.907811 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 27 15:01:57.907811 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 27 15:01:57.907811 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 27 15:01:57.907811 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application session.commit() May 27 15:01:57.907811 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 27 15:01:57.908721 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 27 15:01:57.908721 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 27 15:01:57.908721 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._prepare_impl() May 27 15:01:57.908721 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 27 15:01:57.908721 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.session.flush() May 27 15:01:57.908721 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 27 15:01:57.908721 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._flush(objects) May 27 15:01:57.908721 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 27 15:01:57.908721 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 27 15:01:57.908721 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 27 15:01:57.908721 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application compat.raise_( May 27 15:01:57.908721 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 15:01:57.908721 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception May 27 15:01:57.908721 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 27 15:01:57.908721 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application flush_context.execute() May 27 15:01:57.908721 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 27 15:01:57.908721 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rec.execute(self) May 27 15:01:57.909636 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 27 15:01:57.909636 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 27 15:01:57.909636 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 27 15:01:57.909636 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application _emit_insert_statements( May 27 15:01:57.909636 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 27 15:01:57.909636 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application c = connection._execute_20( May 27 15:01:57.909636 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 27 15:01:57.909636 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 27 15:01:57.909636 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 27 15:01:57.909636 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 27 15:01:57.909636 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 27 15:01:57.909636 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ret = self._execute_context( May 27 15:01:57.909636 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 27 15:01:57.909636 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 27 15:01:57.909636 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 27 15:01:57.910450 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 27 15:01:57.910450 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 15:01:57.910450 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception May 27 15:01:57.910450 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 15:01:57.910450 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 15:01:57.910450 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 15:01:57.910450 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 15:01:57.910450 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 15:01:57.910450 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result = self._query(query) May 27 15:01:57.910450 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 15:01:57.910450 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application conn.query(q) May 27 15:01:57.910450 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 15:01:57.910450 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 15:01:57.910450 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 15:01:57.910450 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result.read() May 27 15:01:57.910450 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 15:01:57.910450 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 15:01:57.915124 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 15:01:57.915124 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 15:01:57.915124 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 15:01:57.915124 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 15:01:57.915124 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 15:01:57.915124 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 15:01:57.915124 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 27 15:01:57.915124 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 27 15:01:57.915124 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [parameters: {'id': '5f62ad2d08cb408aae9d2271812d9c0f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 27 15:01:57.915124 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 27 15:01:57.915124 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:57.915124 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 15:01:57.915124 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:57.915124 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:57.915124 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 27 15:01:57.915963 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 27 15:01:57.915963 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 27 15:01:57.915963 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 27 15:01:57.915963 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 27 15:01:57.915963 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:57.915963 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 15:01:57.915963 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:57.915963 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:57.915963 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:57.915963 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:57.915963 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:57.915963 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:57.915963 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:57.915963 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:57.915963 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:57.915963 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:57.915963 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:57.916883 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:57.916883 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 27 15:01:57.916883 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 27 15:01:57.916883 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:57.916883 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:57.916883 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 27 15:01:57.916883 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.Conflict( May 27 15:01:57.916883 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 5f62ad2d08cb408aae9d2271812d9c0f. May 27 15:01:57.916883 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:57.916883 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 481/938] 10.222.0.50 () {60 vars in 1140 bytes} [Fri May 27 15:01:57 2022] POST /identity/v3/projects => generated 262 bytes in 198 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 27 15:01:57.916883 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8682c8f2-9fa2-4293-accb-ac09f40a9b57 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:57.926905 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872951b0-3d7c-493f-9cce-e2e360fbcb74 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:57.958782 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8a86ba-ec91-47b0-a7a8-79a5d5d68428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:57.959678 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a86ba-ec91-47b0-a7a8-79a5d5d68428 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:57.960036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a86ba-ec91-47b0-a7a8-79a5d5d68428 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:57.960473 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8a86ba-ec91-47b0-a7a8-79a5d5d68428 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:57.972454 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8a86ba-ec91-47b0-a7a8-79a5d5d68428 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:57.974079 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8a86ba-ec91-47b0-a7a8-79a5d5d68428 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:57.993228 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-dd8a86ba-ec91-47b0-a7a8-79a5d5d68428 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:57.993228 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:57.993228 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:57.993228 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:57.993228 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:57.993228 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:57.993228 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:57.993228 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:57.993228 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:57.993228 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:57.993228 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:57.993228 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:57.993228 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:57.993228 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:57.993228 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:57.994186 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:57.994186 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:57.994186 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:57.994186 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:57.994186 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:57.994186 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:57.994186 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:57.994186 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:57.994186 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:57.994186 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:57.994186 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:57.994186 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:57.994186 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:57.994186 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:57.994186 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:57.994186 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:57.994186 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:57.995126 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:57.995126 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:57.995126 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:57.995126 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:57.999382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 458/939] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:01:58.001035 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8a86ba-ec91-47b0-a7a8-79a5d5d68428 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:58.055838 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8682c8f2-9fa2-4293-accb-ac09f40a9b57 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:58.096354 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78219946-dfa8-4dea-a1d7-4284e79fe3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:58.097166 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78219946-dfa8-4dea-a1d7-4284e79fe3a5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:58.097513 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78219946-dfa8-4dea-a1d7-4284e79fe3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:58.097854 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78219946-dfa8-4dea-a1d7-4284e79fe3a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:58.105380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78219946-dfa8-4dea-a1d7-4284e79fe3a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:58.107207 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78219946-dfa8-4dea-a1d7-4284e79fe3a5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:58.120111 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8a86ba-ec91-47b0-a7a8-79a5d5d68428 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:58.124089 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-78219946-dfa8-4dea-a1d7-4284e79fe3a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:58.124089 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:58.124089 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:58.124089 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:58.124089 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:58.124089 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:58.124089 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:58.124089 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:58.124089 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:58.124089 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:58.124089 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:58.124089 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:58.124089 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 15:01:58.124089 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 15:01:58.124089 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 15:01:58.124991 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 15:01:58.124991 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:58.124991 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:58.124991 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:58.124991 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:58.124991 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:58.124991 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:58.124991 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:58.124991 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:58.124991 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:58.124991 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:58.124991 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:58.124991 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:58.124991 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:58.124991 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:58.124991 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:58.124991 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:58.125905 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 15:01:58.125905 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 15:01:58.125905 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 15:01:58.125905 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 15:01:58.125905 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 15:01:58.125905 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 15:01:58.125905 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:58.125905 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:58.130041 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 482/940] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:01:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:58.131037 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78219946-dfa8-4dea-a1d7-4284e79fe3a5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:58.166070 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e808ff-a5dc-4732-818b-20b1e3212054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:58.167346 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e808ff-a5dc-4732-818b-20b1e3212054 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:58.167779 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e808ff-a5dc-4732-818b-20b1e3212054 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:58.168475 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e808ff-a5dc-4732-818b-20b1e3212054 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:58.169774 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e808ff-a5dc-4732-818b-20b1e3212054 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:58.170352 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e808ff-a5dc-4732-818b-20b1e3212054 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:58.172703 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e808ff-a5dc-4732-818b-20b1e3212054 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:58.194894 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 459/941] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:01:58.197041 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e808ff-a5dc-4732-818b-20b1e3212054 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:58.211647 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:58.214443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:58.272809 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78219946-dfa8-4dea-a1d7-4284e79fe3a5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:58.316348 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e808ff-a5dc-4732-818b-20b1e3212054 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:58.317990 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a71d0-9b5f-4544-94c4-dfc0e52d171c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:58.318918 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a71d0-9b5f-4544-94c4-dfc0e52d171c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:58.319210 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a71d0-9b5f-4544-94c4-dfc0e52d171c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:58.319490 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a71d0-9b5f-4544-94c4-dfc0e52d171c None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:58.320293 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05a71d0-9b5f-4544-94c4-dfc0e52d171c None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:58.321556 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05a71d0-9b5f-4544-94c4-dfc0e52d171c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:58.323082 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05a71d0-9b5f-4544-94c4-dfc0e52d171c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:58.355034 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 483/942] 10.222.0.50 () {56 vars in 1112 bytes} [Fri May 27 15:01:58 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:58.364858 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0c16c0-39b3-4298-a0e5-fc0cbccb23b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:58.366112 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c16c0-39b3-4298-a0e5-fc0cbccb23b2 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:58.366649 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c16c0-39b3-4298-a0e5-fc0cbccb23b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:58.367172 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c16c0-39b3-4298-a0e5-fc0cbccb23b2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:58.376595 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0c16c0-39b3-4298-a0e5-fc0cbccb23b2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:58.378591 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0c16c0-39b3-4298-a0e5-fc0cbccb23b2 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:58.397432 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-db0c16c0-39b3-4298-a0e5-fc0cbccb23b2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 15:01:58.397432 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:58.397432 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:58.397432 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:58.397432 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:58.397432 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:58.397432 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:58.397432 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:58.397432 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:58.397432 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:58.397432 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:58.397432 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:58.397432 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 15:01:58.397432 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 15:01:58.397432 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 15:01:58.397432 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 15:01:58.398452 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:58.398452 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:58.398452 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 15:01:58.398452 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:58.398452 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 15:01:58.398452 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:01:58.398452 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:58.398452 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:58.398452 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:58.398452 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:58.398452 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:58.398452 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:58.398452 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:58.398452 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:58.398452 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:58.398452 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:58.398452 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:58.398452 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:58.399448 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:58.399448 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:58.399448 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 15:01:58.399448 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 15:01:58.399448 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 15:01:58.399448 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 15:01:58.399448 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 15:01:58.399448 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 15:01:58.399448 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 15:01:58.399448 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:58.402367 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 460/943] 10.222.0.50 () {56 vars in 1117 bytes} [Fri May 27 15:01:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:01:58.408377 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a71d0-9b5f-4544-94c4-dfc0e52d171c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:58.519352 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a71d0-9b5f-4544-94c4-dfc0e52d171c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:58.528041 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76700d-7948-45db-80bd-251bfcff9298 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:58.528041 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76700d-7948-45db-80bd-251bfcff9298 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:58.528041 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76700d-7948-45db-80bd-251bfcff9298 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:58.531097 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 461/944] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:58.533684 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619763e2-09cb-49fe-8538-c0b7062c137f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:58.533968 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619763e2-09cb-49fe-8538-c0b7062c137f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:58.534245 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619763e2-09cb-49fe-8538-c0b7062c137f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:58.545176 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:58.552243 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-619763e2-09cb-49fe-8538-c0b7062c137f None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:58.553444 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-619763e2-09cb-49fe-8538-c0b7062c137f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:58.564436 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff986e2-a55f-425b-b3b1-45e974b5a7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:58.565135 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff986e2-a55f-425b-b3b1-45e974b5a7af None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:58.565745 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff986e2-a55f-425b-b3b1-45e974b5a7af None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:58.566028 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff986e2-a55f-425b-b3b1-45e974b5a7af None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:58.567214 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff986e2-a55f-425b-b3b1-45e974b5a7af None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:58.567533 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff986e2-a55f-425b-b3b1-45e974b5a7af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:58.569591 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff986e2-a55f-425b-b3b1-45e974b5a7af None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:58.595194 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 484/945] 10.222.0.50 () {56 vars in 1149 bytes} [Fri May 27 15:01:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:01:58.600442 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff986e2-a55f-425b-b3b1-45e974b5a7af None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:58.625192 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:58.633122 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 462/946] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 15:01:58.656625 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619763e2-09cb-49fe-8538-c0b7062c137f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:58.717159 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff986e2-a55f-425b-b3b1-45e974b5a7af None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:58.755587 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3decd381-39d9-4a1d-8580-52943b4df698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:58.756977 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3decd381-39d9-4a1d-8580-52943b4df698 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:58.757722 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3decd381-39d9-4a1d-8580-52943b4df698 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:58.758305 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3decd381-39d9-4a1d-8580-52943b4df698 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:58.770787 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3decd381-39d9-4a1d-8580-52943b4df698 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:58.772670 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3decd381-39d9-4a1d-8580-52943b4df698 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:58.781876 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619763e2-09cb-49fe-8538-c0b7062c137f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:58.792178 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-3decd381-39d9-4a1d-8580-52943b4df698 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:58.792178 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:58.792178 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:58.792178 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:58.792178 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:58.792178 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:58.792178 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:58.792178 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:58.792178 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:58.792178 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:58.792178 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:58.792178 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:58.792178 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:01:58.792178 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:01:58.792178 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:01:58.793286 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:01:58.793286 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:58.793286 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:58.793286 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:58.793286 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:58.793286 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:58.793286 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:58.793286 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:58.793286 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:58.793286 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:58.793286 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:58.793286 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:58.793286 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:58.793286 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:58.793286 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:58.793286 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:58.793286 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:58.794389 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:01:58.794389 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:01:58.794389 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:01:58.794389 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:58.798159 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 485/947] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:01:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:01:58.799608 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3decd381-39d9-4a1d-8580-52943b4df698 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:58.825251 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae90ca-9680-4dd5-a6af-626e93029367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:58.826896 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae90ca-9680-4dd5-a6af-626e93029367 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:58.827962 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae90ca-9680-4dd5-a6af-626e93029367 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:58.828392 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae90ca-9680-4dd5-a6af-626e93029367 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:58.829490 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae90ca-9680-4dd5-a6af-626e93029367 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:58.831772 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae90ca-9680-4dd5-a6af-626e93029367 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:58.837559 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:58.853205 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-1eae90ca-9680-4dd5-a6af-626e93029367 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 15:01:58.853205 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:58.853205 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 15:01:58.853205 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 15:01:58.853205 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 15:01:58.853205 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 15:01:58.853205 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 15:01:58.853205 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:01:58.853205 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 15:01:58.853205 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:58.853205 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 15:01:58.853205 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:58.853205 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:58.853205 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:58.853205 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:58.853205 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:58.853205 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:58.854421 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:58.854421 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:58.854421 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:58.854421 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:58.854421 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:58.854421 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:58.854421 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 15:01:58.854421 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 15:01:58.854421 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 15:01:58.854421 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 15:01:58.854421 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:58.854421 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:58.854421 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:58.854421 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:58.854421 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:58.854421 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:58.855470 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:58.855470 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:01:58.855470 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:58.855470 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:58.855470 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:58.855470 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:58.855470 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:58.855470 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:58.855470 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:58.855470 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:58.855470 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 15:01:58.855470 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:01:58.855470 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 15:01:58.855470 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:01:58.859582 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 463/948] 10.222.0.50 () {56 vars in 1126 bytes} [Fri May 27 15:01:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:01:58.866892 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae90ca-9680-4dd5-a6af-626e93029367 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:58.932772 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3decd381-39d9-4a1d-8580-52943b4df698 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:58.946486 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:58.977989 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d59e6c-0e8d-4838-a133-71e49666f135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:58.978828 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d59e6c-0e8d-4838-a133-71e49666f135 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:58.979257 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d59e6c-0e8d-4838-a133-71e49666f135 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:58.979895 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d59e6c-0e8d-4838-a133-71e49666f135 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:58.981103 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d59e6c-0e8d-4838-a133-71e49666f135 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:58.981507 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d59e6c-0e8d-4838-a133-71e49666f135 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:58.983372 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d59e6c-0e8d-4838-a133-71e49666f135 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:58.999349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 486/949] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:01:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:01:59.002627 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae90ca-9680-4dd5-a6af-626e93029367 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:59.006902 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d59e6c-0e8d-4838-a133-71e49666f135 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:59.056640 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36761914-0405-4742-8ae8-254e036236fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:59.057834 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36761914-0405-4742-8ae8-254e036236fb None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:59.058638 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36761914-0405-4742-8ae8-254e036236fb None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:59.059087 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36761914-0405-4742-8ae8-254e036236fb None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:59.059943 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36761914-0405-4742-8ae8-254e036236fb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:59.060398 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36761914-0405-4742-8ae8-254e036236fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:59.063401 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36761914-0405-4742-8ae8-254e036236fb None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:59.071651 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 464/950] 10.222.0.50 () {56 vars in 1122 bytes} [Fri May 27 15:01:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:59.076448 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36761914-0405-4742-8ae8-254e036236fb None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:59.119817 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:59.134024 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d59e6c-0e8d-4838-a133-71e49666f135 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:59.182522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3841fbd6-f551-4afe-9315-12885182529c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:59.183365 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3841fbd6-f551-4afe-9315-12885182529c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:59.183647 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3841fbd6-f551-4afe-9315-12885182529c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:59.183923 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3841fbd6-f551-4afe-9315-12885182529c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:59.194364 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3841fbd6-f551-4afe-9315-12885182529c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:59.196090 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3841fbd6-f551-4afe-9315-12885182529c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:59.199851 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36761914-0405-4742-8ae8-254e036236fb None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:59.215409 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-3841fbd6-f551-4afe-9315-12885182529c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:59.215409 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:01:59.215409 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:01:59.215409 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:01:59.215409 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:01:59.215409 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:01:59.215409 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:01:59.215409 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:01:59.215409 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:01:59.215409 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:01:59.215409 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:01:59.215409 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:01:59.215409 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 15:01:59.215409 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 15:01:59.215409 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 15:01:59.216503 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 15:01:59.216503 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:01:59.216503 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:01:59.216503 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:01:59.216503 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:01:59.216503 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:01:59.216503 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:01:59.216503 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:01:59.216503 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:01:59.216503 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:01:59.216503 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:01:59.216503 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:01:59.216503 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:01:59.216503 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:01:59.216503 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:01:59.216503 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:01:59.216503 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:01:59.217558 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 15:01:59.217558 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 15:01:59.217558 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 15:01:59.217558 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 15:01:59.217558 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 15:01:59.217558 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 15:01:59.217558 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:01:59.217558 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:01:59.221208 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 487/951] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:01:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:01:59.227417 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3841fbd6-f551-4afe-9315-12885182529c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:59.236854 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:59.247202 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a67134-fe4a-4a55-8962-d516f9dad77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:59.248712 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a67134-fe4a-4a55-8962-d516f9dad77f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:59.248941 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a67134-fe4a-4a55-8962-d516f9dad77f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:59.249144 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a67134-fe4a-4a55-8962-d516f9dad77f None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:59.249854 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a67134-fe4a-4a55-8962-d516f9dad77f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:59.250486 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a67134-fe4a-4a55-8962-d516f9dad77f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:59.251721 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a67134-fe4a-4a55-8962-d516f9dad77f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:59.261509 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 465/952] 10.222.0.50 () {56 vars in 1122 bytes} [Fri May 27 15:01:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:01:59.262639 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a67134-fe4a-4a55-8962-d516f9dad77f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:59.359750 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3841fbd6-f551-4afe-9315-12885182529c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:59.385330 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a67134-fe4a-4a55-8962-d516f9dad77f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:59.405250 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d27c7e-4caf-432e-b37b-f6e4bfdefd93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:59.406428 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d27c7e-4caf-432e-b37b-f6e4bfdefd93 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:59.406811 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d27c7e-4caf-432e-b37b-f6e4bfdefd93 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:59.407221 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d27c7e-4caf-432e-b37b-f6e4bfdefd93 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:59.408117 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d27c7e-4caf-432e-b37b-f6e4bfdefd93 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:01:59.408527 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d27c7e-4caf-432e-b37b-f6e4bfdefd93 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:59.410589 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d27c7e-4caf-432e-b37b-f6e4bfdefd93 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:59.426446 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79be04b-a8ba-4e16-a262-2f1c9ba6d444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:59.431677 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 488/953] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 15:01:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:01:59.431847 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79be04b-a8ba-4e16-a262-2f1c9ba6d444 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:59.431847 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79be04b-a8ba-4e16-a262-2f1c9ba6d444 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:59.431847 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79be04b-a8ba-4e16-a262-2f1c9ba6d444 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:59.432543 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79be04b-a8ba-4e16-a262-2f1c9ba6d444 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:59.433874 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79be04b-a8ba-4e16-a262-2f1c9ba6d444 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:59.436295 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d27c7e-4caf-432e-b37b-f6e4bfdefd93 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:59.450572 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:59.466220 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 466/954] 10.222.0.50 () {60 vars in 1144 bytes} [Fri May 27 15:01:59 2022] POST /identity/v3/endpoints => generated 349 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:01:59.562366 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d27c7e-4caf-432e-b37b-f6e4bfdefd93 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:01:59.612652 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc060de3-f5a2-44f5-9f51-0d4e149e9709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:01:59.613942 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc060de3-f5a2-44f5-9f51-0d4e149e9709 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:59.614476 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc060de3-f5a2-44f5-9f51-0d4e149e9709 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:59.614996 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc060de3-f5a2-44f5-9f51-0d4e149e9709 None admin] PATH_INFO: `/v3/projects/daf12585428b41a2b45bdbc85413a6c4/users/09c23d1c732c452a9597c647ad0241f0/roles/4564db14e2a543f18cb88a5bea5aada2` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:59.617648 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc060de3-f5a2-44f5-9f51-0d4e149e9709 None admin] RBAC: Authorizing `identity:create_grant(project_id=daf12585428b41a2b45bdbc85413a6c4, user_id=09c23d1c732c452a9597c647ad0241f0, role_id=4564db14e2a543f18cb88a5bea5aada2)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:01:59.619351 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc060de3-f5a2-44f5-9f51-0d4e149e9709 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:01:59.637815 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 489/955] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 15:01:59 2022] PUT /identity/v3/projects/daf12585428b41a2b45bdbc85413a6c4/users/09c23d1c732c452a9597c647ad0241f0/roles/4564db14e2a543f18cb88a5bea5aada2 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:01:59.724795 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599bf08-23bd-4d58-b590-cc12aac1479c None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:59.725143 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599bf08-23bd-4d58-b590-cc12aac1479c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:59.725370 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599bf08-23bd-4d58-b590-cc12aac1479c None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:59.727110 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 467/956] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:01:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:01:59.733691 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6803b61a-3535-44a7-84de-a82dcc5dd5e2 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:01:59.734665 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6803b61a-3535-44a7-84de-a82dcc5dd5e2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:01:59.735205 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6803b61a-3535-44a7-84de-a82dcc5dd5e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:01:59.754699 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6803b61a-3535-44a7-84de-a82dcc5dd5e2 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:01:59.756619 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6803b61a-3535-44a7-84de-a82dcc5dd5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:01:59.825673 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:59.834774 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 490/957] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:01:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:01:59.859255 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0599bf08-23bd-4d58-b590-cc12aac1479c None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:01:59.919482 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:01:59.980883 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0599bf08-23bd-4d58-b590-cc12aac1479c None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:00.024332 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fec010-58f9-4805-a330-d7a847f224b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:00.025521 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fec010-58f9-4805-a330-d7a847f224b6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:00.026007 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fec010-58f9-4805-a330-d7a847f224b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:00.026483 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fec010-58f9-4805-a330-d7a847f224b6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:00.027395 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fec010-58f9-4805-a330-d7a847f224b6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:00.029032 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fec010-58f9-4805-a330-d7a847f224b6 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:00.043862 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-97fec010-58f9-4805-a330-d7a847f224b6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 27 15:02:00.043862 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:00.043862 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 15:02:00.043862 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 15:02:00.043862 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 15:02:00.043862 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 15:02:00.043862 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 15:02:00.043862 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:02:00.043862 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 15:02:00.043862 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:02:00.043862 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 15:02:00.043862 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:02:00.043862 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:00.043862 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:02:00.043862 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:02:00.043862 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:02:00.043862 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:02:00.045068 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:02:00.045068 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:02:00.045068 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:02:00.045068 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:02:00.045068 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:02:00.045068 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:02:00.045068 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 15:02:00.045068 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 15:02:00.045068 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 15:02:00.045068 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 15:02:00.045068 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:02:00.045068 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:02:00.045068 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:02:00.045068 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:02:00.045068 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:02:00.045068 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:02:00.046170 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:02:00.046170 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:02:00.046170 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:02:00.046170 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:02:00.046170 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:02:00.046170 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:02:00.046170 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:02:00.046170 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:02:00.046170 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:02:00.046170 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:02:00.046170 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 15:02:00.046170 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:02:00.046170 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 15:02:00.046170 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:02:00.049187 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 468/958] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:01:59 2022] GET /identity/v3/services/compute => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:02:00.056189 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6803b61a-3535-44a7-84de-a82dcc5dd5e2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:00.185978 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6803b61a-3535-44a7-84de-a82dcc5dd5e2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:00.225871 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00807fa1-3df7-495b-9153-b720b315f32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:00.226574 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00807fa1-3df7-495b-9153-b720b315f32a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:00.226966 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00807fa1-3df7-495b-9153-b720b315f32a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:00.227466 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00807fa1-3df7-495b-9153-b720b315f32a None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:00.228702 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00807fa1-3df7-495b-9153-b720b315f32a None admin] RBAC: Adding query filter params (name=compute) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:00.229020 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00807fa1-3df7-495b-9153-b720b315f32a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:00.230407 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00807fa1-3df7-495b-9153-b720b315f32a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:00.236520 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 491/959] 10.222.0.50 () {56 vars in 1112 bytes} [Fri May 27 15:02:00 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:00.244670 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fec010-58f9-4805-a330-d7a847f224b6 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:00.371702 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fec010-58f9-4805-a330-d7a847f224b6 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:00.383893 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:00.412620 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cf2f31-4bcf-4bb3-9a8a-60ae16633a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:00.413797 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf2f31-4bcf-4bb3-9a8a-60ae16633a80 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:00.414360 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf2f31-4bcf-4bb3-9a8a-60ae16633a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:00.414915 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cf2f31-4bcf-4bb3-9a8a-60ae16633a80 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:00.415825 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cf2f31-4bcf-4bb3-9a8a-60ae16633a80 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:00.416330 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cf2f31-4bcf-4bb3-9a8a-60ae16633a80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:00.417910 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cf2f31-4bcf-4bb3-9a8a-60ae16633a80 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:00.424020 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 469/960] 10.222.0.50 () {56 vars in 1112 bytes} [Fri May 27 15:02:00 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:00.987638 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968523f1-b3bf-41de-bd69-b241f93ec979 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:00.988156 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968523f1-b3bf-41de-bd69-b241f93ec979 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:00.988262 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968523f1-b3bf-41de-bd69-b241f93ec979 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:00.989963 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 492/961] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:02:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:02:00.995321 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22957e0b-5c80-4034-a134-104b5d950722 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:00.995743 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22957e0b-5c80-4034-a134-104b5d950722 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:00.995927 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22957e0b-5c80-4034-a134-104b5d950722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:01.013130 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-22957e0b-5c80-4034-a134-104b5d950722 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:02:01.014544 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-22957e0b-5c80-4034-a134-104b5d950722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:02:01.086248 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 470/962] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:02:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:02:01.111591 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968523f1-b3bf-41de-bd69-b241f93ec979 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:01.179044 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:01.226998 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968523f1-b3bf-41de-bd69-b241f93ec979 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:01.263243 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e240dfde-cd64-42e1-9190-6cb3078dc4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:01.263909 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240dfde-cd64-42e1-9190-6cb3078dc4c1 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:01.264199 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240dfde-cd64-42e1-9190-6cb3078dc4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:01.264477 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240dfde-cd64-42e1-9190-6cb3078dc4c1 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:01.265177 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e240dfde-cd64-42e1-9190-6cb3078dc4c1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:01.266611 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e240dfde-cd64-42e1-9190-6cb3078dc4c1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:01.286465 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.sql.core [None req-e240dfde-cd64-42e1-9190-6cb3078dc4c1 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 27 15:02:01.286465 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 27 15:02:01.286465 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [parameters: {'user_id': '8dbe92615e1e41c7923fc71406891f8e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 27 15:02:01.286465 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80395) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 27 15:02:01.293964 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-e240dfde-cd64-42e1-9190-6cb3078dc4c1 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 27 15:02:01.293964 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:01.293964 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 15:02:01.293964 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 15:02:01.293964 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 15:02:01.293964 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 15:02:01.293964 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 15:02:01.293964 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result = self._query(query) May 27 15:02:01.293964 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 15:02:01.293964 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application conn.query(q) May 27 15:02:01.293964 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 15:02:01.293964 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 15:02:01.293964 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 15:02:01.293964 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result.read() May 27 15:02:01.293964 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 15:02:01.295479 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 15:02:01.295479 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 15:02:01.295479 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 15:02:01.295479 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 15:02:01.295479 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 15:02:01.295479 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 15:02:01.295479 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 15:02:01.295479 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 27 15:02:01.295479 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:02:01.295479 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 27 15:02:01.295479 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:02:01.295479 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:01.295479 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 27 15:02:01.295479 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 27 15:02:01.295479 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 27 15:02:01.295479 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 27 15:02:01.295479 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 15:02:01.295479 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application next(self.gen) May 27 15:02:01.296642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 27 15:02:01.296642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application yield resource May 27 15:02:01.296642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 27 15:02:01.296642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application next(self.gen) May 27 15:02:01.296642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 27 15:02:01.296642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.session.rollback() May 27 15:02:01.296642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 27 15:02:01.296642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.force_reraise() May 27 15:02:01.296642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 27 15:02:01.296642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise self.value May 27 15:02:01.296642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 27 15:02:01.296642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 27 15:02:01.296642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 27 15:02:01.296642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application session.commit() May 27 15:02:01.296642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 27 15:02:01.296642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 27 15:02:01.296642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 27 15:02:01.298922 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._prepare_impl() May 27 15:02:01.298922 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 27 15:02:01.298922 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.session.flush() May 27 15:02:01.298922 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 27 15:02:01.298922 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._flush(objects) May 27 15:02:01.298922 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 27 15:02:01.298922 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 27 15:02:01.298922 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 27 15:02:01.298922 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application compat.raise_( May 27 15:02:01.298922 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 15:02:01.298922 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception May 27 15:02:01.298922 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 27 15:02:01.298922 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application flush_context.execute() May 27 15:02:01.298922 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 27 15:02:01.298922 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rec.execute(self) May 27 15:02:01.298922 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 27 15:02:01.298922 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 27 15:02:01.303936 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 27 15:02:01.303936 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application _emit_insert_statements( May 27 15:02:01.303936 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 27 15:02:01.303936 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result = connection._execute_20( May 27 15:02:01.303936 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 27 15:02:01.303936 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 27 15:02:01.303936 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 27 15:02:01.303936 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 27 15:02:01.303936 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 27 15:02:01.303936 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ret = self._execute_context( May 27 15:02:01.303936 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 27 15:02:01.303936 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 27 15:02:01.303936 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 27 15:02:01.303936 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 27 15:02:01.303936 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 27 15:02:01.303936 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception May 27 15:02:01.304847 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 27 15:02:01.304847 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.dialect.do_execute( May 27 15:02:01.304847 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 27 15:02:01.304847 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 27 15:02:01.304847 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 27 15:02:01.304847 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result = self._query(query) May 27 15:02:01.304847 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 27 15:02:01.304847 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application conn.query(q) May 27 15:02:01.304847 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 27 15:02:01.304847 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 27 15:02:01.304847 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 27 15:02:01.304847 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result.read() May 27 15:02:01.304847 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 27 15:02:01.304847 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 27 15:02:01.304847 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 27 15:02:01.304847 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application packet.raise_for_error() May 27 15:02:01.305739 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 27 15:02:01.305739 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 27 15:02:01.305739 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 27 15:02:01.305739 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 27 15:02:01.305739 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 27 15:02:01.305739 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 27 15:02:01.305739 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [parameters: {'user_id': '8dbe92615e1e41c7923fc71406891f8e', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 27 15:02:01.305739 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 27 15:02:01.305739 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:02:01.305739 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 15:02:01.305739 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:02:01.305739 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:01.305739 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:02:01.305739 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:02:01.305739 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:02:01.306539 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:02:01.306539 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:02:01.306539 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:02:01.306539 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:02:01.306539 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:02:01.306539 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:02:01.306539 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:02:01.306539 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 27 15:02:01.306539 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 27 15:02:01.306539 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:02:01.306539 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:02:01.306539 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 15:02:01.306539 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:02:01.306539 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 15:02:01.306539 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:02:01.306539 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 27 15:02:01.306539 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 27 15:02:01.307487 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 27 15:02:01.307487 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 27 15:02:01.307487 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 27 15:02:01.307487 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 27 15:02:01.307487 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 27 15:02:01.307487 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:02:01.307487 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 493/963] 10.222.0.50 () {60 vars in 1131 bytes} [Fri May 27 15:02:01 2022] POST /identity/v3/users => generated 160 bytes in 185 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 27 15:02:01.307905 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22957e0b-5c80-4034-a134-104b5d950722 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:01.413919 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22957e0b-5c80-4034-a134-104b5d950722 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:01.416812 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:01.450140 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f2115d-48e1-4b6e-a452-2a9afc7fb7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:01.452001 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2115d-48e1-4b6e-a452-2a9afc7fb7bd None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:01.452601 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2115d-48e1-4b6e-a452-2a9afc7fb7bd None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:01.453096 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f2115d-48e1-4b6e-a452-2a9afc7fb7bd None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:01.460772 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f2115d-48e1-4b6e-a452-2a9afc7fb7bd None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:01.462531 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f2115d-48e1-4b6e-a452-2a9afc7fb7bd None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:01.478389 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-06f2115d-48e1-4b6e-a452-2a9afc7fb7bd None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 27 15:02:01.478389 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:01.478389 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:02:01.478389 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:02:01.478389 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:02:01.478389 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:02:01.478389 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:02:01.478389 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:02:01.478389 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:02:01.478389 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:02:01.478389 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:02:01.478389 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:02:01.478389 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 15:02:01.478389 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 15:02:01.478389 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 15:02:01.478389 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 15:02:01.479482 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:02:01.479482 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:02:01.479482 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 15:02:01.479482 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:02:01.479482 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 15:02:01.479482 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:02:01.479482 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:02:01.479482 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:02:01.479482 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:02:01.479482 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:02:01.479482 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:02:01.479482 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:02:01.479482 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:02:01.479482 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:02:01.479482 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:02:01.479482 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:02:01.479482 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:02:01.479482 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:02:01.480679 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:02:01.480679 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:02:01.480679 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 15:02:01.480679 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 15:02:01.480679 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 15:02:01.480679 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 15:02:01.480679 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 15:02:01.480679 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 15:02:01.480679 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 27 15:02:01.480679 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:02:01.486944 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 471/964] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:02:01 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:02:01.489388 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e240dfde-cd64-42e1-9190-6cb3078dc4c1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:01.596170 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e240dfde-cd64-42e1-9190-6cb3078dc4c1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:01.620607 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:01.637926 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa0ffd7-4831-4702-b0bf-0d0c98ebcb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:01.638685 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0ffd7-4831-4702-b0bf-0d0c98ebcb64 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:01.639091 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0ffd7-4831-4702-b0bf-0d0c98ebcb64 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:01.639315 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa0ffd7-4831-4702-b0bf-0d0c98ebcb64 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:01.640157 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa0ffd7-4831-4702-b0bf-0d0c98ebcb64 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:01.640650 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa0ffd7-4831-4702-b0bf-0d0c98ebcb64 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:01.642367 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa0ffd7-4831-4702-b0bf-0d0c98ebcb64 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:01.694987 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 494/965] 10.222.0.50 () {56 vars in 1101 bytes} [Fri May 27 15:02:01 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:01.834137 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49408b0d-2777-48ad-b37f-d5d821465665 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:01.834463 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49408b0d-2777-48ad-b37f-d5d821465665 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:01.834696 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49408b0d-2777-48ad-b37f-d5d821465665 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:01.836421 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 472/966] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:02:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:02:01.843456 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef4d83-8703-4382-b771-3856bf328a82 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:01.843792 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef4d83-8703-4382-b771-3856bf328a82 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:01.844024 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ef4d83-8703-4382-b771-3856bf328a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:01.863749 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f7ef4d83-8703-4382-b771-3856bf328a82 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:02:01.865215 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f7ef4d83-8703-4382-b771-3856bf328a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:02:01.941812 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 495/967] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:02:01 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:02:01.965515 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49408b0d-2777-48ad-b37f-d5d821465665 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:02.070837 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49408b0d-2777-48ad-b37f-d5d821465665 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:02.074292 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74152d22-f99a-4413-ab2c-4392a5dfb3bd None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:02.074685 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74152d22-f99a-4413-ab2c-4392a5dfb3bd None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:02.074908 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74152d22-f99a-4413-ab2c-4392a5dfb3bd None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:02.076930 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 496/968] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:02:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 15:02:02.081728 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28137f-5596-41f9-b8cf-92ffe0199e46 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:02.082017 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28137f-5596-41f9-b8cf-92ffe0199e46 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:02.082299 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28137f-5596-41f9-b8cf-92ffe0199e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:02.101746 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4f28137f-5596-41f9-b8cf-92ffe0199e46 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:02:02.102970 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4f28137f-5596-41f9-b8cf-92ffe0199e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:02:02.109029 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb34d7e-9e54-4202-b83b-b13b8b1ad766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:02.109841 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb34d7e-9e54-4202-b83b-b13b8b1ad766 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:02.110124 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb34d7e-9e54-4202-b83b-b13b8b1ad766 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:02.110400 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb34d7e-9e54-4202-b83b-b13b8b1ad766 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:02.111082 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb34d7e-9e54-4202-b83b-b13b8b1ad766 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:02.112447 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb34d7e-9e54-4202-b83b-b13b8b1ad766 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:02.126042 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-7bb34d7e-9e54-4202-b83b-b13b8b1ad766 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 15:02:02.126042 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:02.126042 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 15:02:02.126042 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 15:02:02.126042 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 15:02:02.126042 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 15:02:02.126042 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 15:02:02.126042 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:02:02.126042 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 15:02:02.126042 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:02:02.126042 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 15:02:02.126042 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:02:02.126042 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:02.126042 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:02:02.126042 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:02:02.126042 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:02:02.126042 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:02:02.127120 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:02:02.127120 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:02:02.127120 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:02:02.127120 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:02:02.127120 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:02:02.127120 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:02:02.127120 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 15:02:02.127120 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 15:02:02.127120 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 15:02:02.127120 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 15:02:02.127120 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:02:02.127120 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:02:02.127120 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:02:02.127120 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:02:02.127120 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:02:02.127120 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:02:02.132945 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:02:02.132945 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:02:02.132945 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:02:02.132945 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:02:02.132945 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:02:02.132945 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:02:02.132945 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:02:02.132945 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:02:02.132945 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:02:02.132945 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:02:02.132945 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 15:02:02.132945 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:02:02.132945 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 15:02:02.132945 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:02:02.132945 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 473/969] 10.222.0.50 () {56 vars in 1126 bytes} [Fri May 27 15:02:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:02:02.133752 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb34d7e-9e54-4202-b83b-b13b8b1ad766 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:02.170398 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:02.189767 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 497/970] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:02:02 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 15:02:02.213995 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f28137f-5596-41f9-b8cf-92ffe0199e46 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:02.249084 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb34d7e-9e54-4202-b83b-b13b8b1ad766 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:02.257198 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:02.287285 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0806ecf5-1115-455b-a2d8-fc9ec47554ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:02.288418 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0806ecf5-1115-455b-a2d8-fc9ec47554ca None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:02.288809 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0806ecf5-1115-455b-a2d8-fc9ec47554ca None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:02.289094 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0806ecf5-1115-455b-a2d8-fc9ec47554ca None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:02.289803 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0806ecf5-1115-455b-a2d8-fc9ec47554ca None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:02.290113 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0806ecf5-1115-455b-a2d8-fc9ec47554ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:02.291511 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0806ecf5-1115-455b-a2d8-fc9ec47554ca None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:02.301253 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 474/971] 10.222.0.50 () {56 vars in 1122 bytes} [Fri May 27 15:02:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:02.302516 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0806ecf5-1115-455b-a2d8-fc9ec47554ca None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:02.322121 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f28137f-5596-41f9-b8cf-92ffe0199e46 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:02.357917 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5daadb7-b174-46da-b289-f12a745adea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:02.358722 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5daadb7-b174-46da-b289-f12a745adea3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:02.359063 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5daadb7-b174-46da-b289-f12a745adea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:02.359342 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5daadb7-b174-46da-b289-f12a745adea3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:02.369397 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5daadb7-b174-46da-b289-f12a745adea3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:02:02.378096 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-f5daadb7-b174-46da-b289-f12a745adea3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 15:02:02.378096 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:02.378096 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:02:02.378096 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:02:02.378096 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:02:02.378096 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:02:02.378096 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:02:02.378096 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:02:02.378096 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:02:02.378096 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:02:02.378096 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:02:02.378096 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:02:02.378096 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:02:02.378096 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:02:02.378096 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:02:02.378096 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:02:02.379091 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:02:02.379091 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:02:02.379091 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:02:02.379091 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:02:02.379091 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:02:02.379091 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:02:02.379091 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:02:02.379091 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:02:02.379091 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:02:02.379091 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:02:02.379091 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:02:02.379091 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:02:02.379091 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:02:02.379091 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:02:02.379091 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:02:02.379091 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:02:02.379942 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:02:02.379942 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:02:02.379942 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:02:02.379942 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:02:02.379942 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:02:02.379942 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:02:02.379942 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:02:02.379942 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:02:02.379942 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:02:02.379942 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:02:02.379942 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 15:02:02.379942 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:02:02.383010 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 498/972] 10.222.0.50 () {56 vars in 1120 bytes} [Fri May 27 15:02:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:02:02.385180 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5daadb7-b174-46da-b289-f12a745adea3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:02.411373 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:02.422379 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0806ecf5-1115-455b-a2d8-fc9ec47554ca None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:02.458681 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22ca8de-e5e3-4ec8-8b8a-e9ce892ffa10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:02.459390 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ca8de-e5e3-4ec8-8b8a-e9ce892ffa10 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:02.459681 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ca8de-e5e3-4ec8-8b8a-e9ce892ffa10 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:02.459962 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ca8de-e5e3-4ec8-8b8a-e9ce892ffa10 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:02.460666 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22ca8de-e5e3-4ec8-8b8a-e9ce892ffa10 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:02.460974 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22ca8de-e5e3-4ec8-8b8a-e9ce892ffa10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:02.462376 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22ca8de-e5e3-4ec8-8b8a-e9ce892ffa10 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:02.472654 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 475/973] 10.222.0.50 () {56 vars in 1122 bytes} [Fri May 27 15:02:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:02:02.473724 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22ca8de-e5e3-4ec8-8b8a-e9ce892ffa10 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:02.494825 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:02.498876 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5daadb7-b174-46da-b289-f12a745adea3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:02.537378 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a7066-7666-44d8-8f92-51dc91e35c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:02.538046 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a7066-7666-44d8-8f92-51dc91e35c15 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:02.538359 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a7066-7666-44d8-8f92-51dc91e35c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:02.538608 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a7066-7666-44d8-8f92-51dc91e35c15 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:02.539526 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a7066-7666-44d8-8f92-51dc91e35c15 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:02.539705 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a7066-7666-44d8-8f92-51dc91e35c15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:02.541041 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a7066-7666-44d8-8f92-51dc91e35c15 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:02.550296 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 499/974] 10.222.0.50 () {56 vars in 1115 bytes} [Fri May 27 15:02:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:02:02.555816 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a7066-7666-44d8-8f92-51dc91e35c15 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:02.582579 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22ca8de-e5e3-4ec8-8b8a-e9ce892ffa10 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:02.628792 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fb8994-7f48-40e8-8559-88f8ac854878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:02.629399 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fb8994-7f48-40e8-8559-88f8ac854878 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:02.629691 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fb8994-7f48-40e8-8559-88f8ac854878 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:02.629971 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fb8994-7f48-40e8-8559-88f8ac854878 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:02.630771 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fb8994-7f48-40e8-8559-88f8ac854878 None admin] RBAC: Adding query filter params (interface=admin, service_id=3ff7d3e34aa04d77b50dfc3d6e932c4f, region_id=RegionOne) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:02.631122 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fb8994-7f48-40e8-8559-88f8ac854878 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:02.632491 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fb8994-7f48-40e8-8559-88f8ac854878 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:02.637923 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 476/975] 10.222.0.50 () {56 vars in 1249 bytes} [Fri May 27 15:02:02 2022] GET /identity/v3/endpoints?service_id=3ff7d3e34aa04d77b50dfc3d6e932c4f&interface=admin®ion_id=RegionOne => generated 193 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:02.643193 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fb8994-7f48-40e8-8559-88f8ac854878 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:02.666276 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:02.669535 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a7066-7666-44d8-8f92-51dc91e35c15 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:02.706638 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3159413-1d0c-4022-af00-54c6bc2a4c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:02.707481 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3159413-1d0c-4022-af00-54c6bc2a4c91 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:02.707761 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3159413-1d0c-4022-af00-54c6bc2a4c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:02.708031 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3159413-1d0c-4022-af00-54c6bc2a4c91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:02.719982 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3159413-1d0c-4022-af00-54c6bc2a4c91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:02.721434 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3159413-1d0c-4022-af00-54c6bc2a4c91 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:02.739625 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-b3159413-1d0c-4022-af00-54c6bc2a4c91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:02:02.739625 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:02.739625 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:02:02.739625 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:02:02.739625 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:02:02.739625 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:02:02.739625 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:02:02.739625 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:02:02.739625 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:02:02.739625 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:02:02.739625 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:02:02.739625 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:02:02.739625 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:02:02.739625 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:02:02.739625 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:02:02.740559 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:02:02.740559 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:02:02.740559 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:02:02.740559 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:02:02.740559 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:02:02.740559 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:02:02.740559 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:02:02.740559 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:02:02.740559 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:02:02.740559 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:02:02.740559 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:02:02.740559 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:02:02.740559 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:02:02.740559 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:02:02.740559 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:02:02.740559 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:02:02.740559 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:02:02.741553 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:02:02.741553 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:02:02.741553 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:02:02.741553 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:02:02.741553 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 500/976] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:02:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:02:02.746473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3159413-1d0c-4022-af00-54c6bc2a4c91 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:02.756592 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fb8994-7f48-40e8-8559-88f8ac854878 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:02.763298 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:02.790450 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6723ba9b-2826-4f0b-8695-8b5bafe1b2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:02.791143 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723ba9b-2826-4f0b-8695-8b5bafe1b2c8 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:02.791426 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723ba9b-2826-4f0b-8695-8b5bafe1b2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:02.791711 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723ba9b-2826-4f0b-8695-8b5bafe1b2c8 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:02.792387 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6723ba9b-2826-4f0b-8695-8b5bafe1b2c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:02.793747 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6723ba9b-2826-4f0b-8695-8b5bafe1b2c8 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:02.810891 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 477/977] 10.222.0.50 () {56 vars in 1087 bytes} [Fri May 27 15:02:02 2022] GET /identity/v3/services => generated 1497 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:02:02.810891 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed88aa-a940-4122-bdda-e85e3ba52a2e None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:02.810891 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed88aa-a940-4122-bdda-e85e3ba52a2e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:02.810891 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed88aa-a940-4122-bdda-e85e3ba52a2e None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:02.810891 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 478/978] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:02:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:02:02.810891 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14511627-5bc6-46b3-8ad3-25aaa40d9a1e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:02.810891 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14511627-5bc6-46b3-8ad3-25aaa40d9a1e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:02.811530 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14511627-5bc6-46b3-8ad3-25aaa40d9a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:02.827852 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-14511627-5bc6-46b3-8ad3-25aaa40d9a1e None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:02:02.829075 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-14511627-5bc6-46b3-8ad3-25aaa40d9a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:02:02.856132 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3159413-1d0c-4022-af00-54c6bc2a4c91 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:02.894876 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b2d33e-1f68-47ad-b703-75de0d0c1273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:02.895461 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b2d33e-1f68-47ad-b703-75de0d0c1273 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:02.895760 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b2d33e-1f68-47ad-b703-75de0d0c1273 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:02.896031 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b2d33e-1f68-47ad-b703-75de0d0c1273 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:02.896762 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b2d33e-1f68-47ad-b703-75de0d0c1273 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:02.897062 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b2d33e-1f68-47ad-b703-75de0d0c1273 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:02.898448 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b2d33e-1f68-47ad-b703-75de0d0c1273 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:02.906894 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 479/979] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:02:02 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:02:02.920880 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 501/980] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:02:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:02:02.922746 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14511627-5bc6-46b3-8ad3-25aaa40d9a1e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:02.929393 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b2d33e-1f68-47ad-b703-75de0d0c1273 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:02.940813 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:03.015805 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:03.036807 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14511627-5bc6-46b3-8ad3-25aaa40d9a1e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:03.043466 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b2d33e-1f68-47ad-b703-75de0d0c1273 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:03.072185 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9179b3f4-bfda-40ea-b16b-a520ff9a1dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:03.072995 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9179b3f4-bfda-40ea-b16b-a520ff9a1dde None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:03.073280 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9179b3f4-bfda-40ea-b16b-a520ff9a1dde None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:03.073558 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9179b3f4-bfda-40ea-b16b-a520ff9a1dde None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:03.079111 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a52001c-0608-4525-9e86-3c39a1842b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:03.079780 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52001c-0608-4525-9e86-3c39a1842b3c None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:03.080064 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52001c-0608-4525-9e86-3c39a1842b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:03.080340 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52001c-0608-4525-9e86-3c39a1842b3c None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:03.081067 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a52001c-0608-4525-9e86-3c39a1842b3c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:03.081529 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9179b3f4-bfda-40ea-b16b-a520ff9a1dde None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:03.082444 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a52001c-0608-4525-9e86-3c39a1842b3c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:03.082987 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9179b3f4-bfda-40ea-b16b-a520ff9a1dde None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:03.099447 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 502/981] 10.222.0.50 () {60 vars in 1141 bytes} [Fri May 27 15:02:02 2022] POST /identity/v3/services => generated 242 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:02:03.099651 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-9179b3f4-bfda-40ea-b16b-a520ff9a1dde None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 15:02:03.099651 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:03.099651 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:02:03.099651 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:02:03.099651 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:02:03.099651 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:02:03.099651 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:02:03.099651 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:02:03.099651 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:02:03.099651 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:02:03.099651 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:02:03.099651 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:02:03.099651 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 15:02:03.099651 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 15:02:03.099651 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 15:02:03.099651 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 15:02:03.100549 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:02:03.100549 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:02:03.100549 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 15:02:03.100549 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:02:03.100549 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 15:02:03.100549 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:02:03.100549 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:02:03.100549 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:02:03.100549 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:02:03.100549 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:02:03.100549 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:02:03.100549 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:02:03.100549 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:02:03.100549 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:02:03.100549 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:02:03.100549 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:02:03.100549 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:02:03.100549 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:02:03.101501 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:02:03.101501 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:02:03.101501 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 15:02:03.101501 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 15:02:03.101501 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 15:02:03.101501 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 15:02:03.101501 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 15:02:03.101501 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 15:02:03.101501 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 27 15:02:03.101501 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:02:03.105584 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 480/982] 10.222.0.50 () {56 vars in 1117 bytes} [Fri May 27 15:02:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:02:03.107568 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a52001c-0608-4525-9e86-3c39a1842b3c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:03.234395 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a52001c-0608-4525-9e86-3c39a1842b3c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:03.266696 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:03.274786 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993784f3-b09a-4b33-976f-5bfc77bc0fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:03.275872 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993784f3-b09a-4b33-976f-5bfc77bc0fab None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:03.276359 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993784f3-b09a-4b33-976f-5bfc77bc0fab None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:03.276885 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993784f3-b09a-4b33-976f-5bfc77bc0fab None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:03.277977 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993784f3-b09a-4b33-976f-5bfc77bc0fab None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:03.278530 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993784f3-b09a-4b33-976f-5bfc77bc0fab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:03.280367 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993784f3-b09a-4b33-976f-5bfc77bc0fab None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:03.299775 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 503/983] 10.222.0.50 () {56 vars in 1149 bytes} [Fri May 27 15:02:03 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:03.305448 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9179b3f4-bfda-40ea-b16b-a520ff9a1dde None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:03.387159 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:03.416674 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9179b3f4-bfda-40ea-b16b-a520ff9a1dde None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:03.452794 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dfd319-d9ae-435b-8948-1c2761382889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:03.453502 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dfd319-d9ae-435b-8948-1c2761382889 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:03.453790 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dfd319-d9ae-435b-8948-1c2761382889 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:03.454065 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dfd319-d9ae-435b-8948-1c2761382889 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:03.521712 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dfd319-d9ae-435b-8948-1c2761382889 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:03.523207 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dfd319-d9ae-435b-8948-1c2761382889 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:03.596605 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:03.624280 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 481/984] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:02:03 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 15:02:03.629763 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993784f3-b09a-4b33-976f-5bfc77bc0fab None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:03.746968 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993784f3-b09a-4b33-976f-5bfc77bc0fab None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:03.788036 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb21e86-9ac9-467a-aa34-e02479d4d85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:03.788833 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb21e86-9ac9-467a-aa34-e02479d4d85d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:03.789117 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb21e86-9ac9-467a-aa34-e02479d4d85d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:03.789391 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb21e86-9ac9-467a-aa34-e02479d4d85d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:03.800044 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb21e86-9ac9-467a-aa34-e02479d4d85d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:03.801476 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb21e86-9ac9-467a-aa34-e02479d4d85d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:03.819822 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-fbb21e86-9ac9-467a-aa34-e02479d4d85d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:02:03.819822 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:03.819822 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:02:03.819822 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:02:03.819822 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:02:03.819822 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:02:03.819822 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:02:03.819822 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:02:03.819822 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:02:03.819822 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:02:03.819822 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:02:03.819822 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:02:03.819822 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:02:03.819822 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:02:03.819822 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:02:03.820758 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:02:03.820758 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:02:03.820758 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:02:03.820758 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:02:03.820758 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:02:03.820758 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:02:03.820758 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:02:03.820758 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:02:03.820758 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:02:03.820758 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:02:03.820758 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:02:03.820758 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:02:03.820758 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:02:03.820758 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:02:03.820758 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:02:03.820758 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:02:03.820758 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:02:03.821655 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:02:03.821655 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:02:03.821655 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:02:03.821655 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:02:03.821655 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 504/985] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:02:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:02:03.828242 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dfd319-d9ae-435b-8948-1c2761382889 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:03.929222 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dfd319-d9ae-435b-8948-1c2761382889 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:03.965632 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4153db-61cd-4055-8e28-a3bbb3dda903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:03.966303 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4153db-61cd-4055-8e28-a3bbb3dda903 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:03.967151 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4153db-61cd-4055-8e28-a3bbb3dda903 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:03.967438 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4153db-61cd-4055-8e28-a3bbb3dda903 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:03.968219 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4153db-61cd-4055-8e28-a3bbb3dda903 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:03.968534 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4153db-61cd-4055-8e28-a3bbb3dda903 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:03.969899 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4153db-61cd-4055-8e28-a3bbb3dda903 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:03.986219 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 482/986] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:02:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:03.987397 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb21e86-9ac9-467a-aa34-e02479d4d85d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:04.030636 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:04.090369 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb21e86-9ac9-467a-aa34-e02479d4d85d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:04.125595 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3750606a-2271-4b68-8191-4bb8c8cac787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:04.126388 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3750606a-2271-4b68-8191-4bb8c8cac787 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:04.126673 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3750606a-2271-4b68-8191-4bb8c8cac787 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:04.126980 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3750606a-2271-4b68-8191-4bb8c8cac787 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:04.133847 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3750606a-2271-4b68-8191-4bb8c8cac787 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:04.135311 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3750606a-2271-4b68-8191-4bb8c8cac787 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:04.150120 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-3750606a-2271-4b68-8191-4bb8c8cac787 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:02:04.150120 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:04.150120 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:02:04.150120 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:02:04.150120 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:02:04.150120 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:02:04.150120 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:02:04.150120 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:02:04.150120 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:02:04.150120 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:02:04.150120 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:02:04.150120 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:02:04.150120 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 15:02:04.150120 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 15:02:04.150120 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 15:02:04.151538 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 15:02:04.151538 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:02:04.151538 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:02:04.151538 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:02:04.151538 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:02:04.151538 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:02:04.151538 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:02:04.151538 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:02:04.151538 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:02:04.151538 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:02:04.151538 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:02:04.151538 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:02:04.151538 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:02:04.151538 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:02:04.151538 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:02:04.151538 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:02:04.151538 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:02:04.152586 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 15:02:04.152586 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 15:02:04.152586 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 15:02:04.152586 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 15:02:04.152586 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 15:02:04.152586 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 15:02:04.152586 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:02:04.152586 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:02:04.157509 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4153db-61cd-4055-8e28-a3bbb3dda903 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:04.157942 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 505/987] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:02:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:02:04.194427 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:04.264120 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4153db-61cd-4055-8e28-a3bbb3dda903 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:04.303749 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d21cc4-2589-41fd-b116-472bc834f111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:04.304341 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d21cc4-2589-41fd-b116-472bc834f111 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:04.304626 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d21cc4-2589-41fd-b116-472bc834f111 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:04.304904 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d21cc4-2589-41fd-b116-472bc834f111 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:04.305693 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d21cc4-2589-41fd-b116-472bc834f111 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:04.306003 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d21cc4-2589-41fd-b116-472bc834f111 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:04.307471 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d21cc4-2589-41fd-b116-472bc834f111 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:04.321472 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 483/988] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 15:02:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:04.330421 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3750606a-2271-4b68-8191-4bb8c8cac787 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:04.427607 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:04.431536 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3750606a-2271-4b68-8191-4bb8c8cac787 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:04.466415 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b723e5d-afd7-49d6-860d-115154dd15f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:04.467745 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b723e5d-afd7-49d6-860d-115154dd15f0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:04.468048 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b723e5d-afd7-49d6-860d-115154dd15f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:04.468314 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b723e5d-afd7-49d6-860d-115154dd15f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:04.469135 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b723e5d-afd7-49d6-860d-115154dd15f0 None admin] RBAC: Adding query filter params (role.id=4564db14e2a543f18cb88a5bea5aada2, scope.project.id=daf12585428b41a2b45bdbc85413a6c4, user.id=09c23d1c732c452a9597c647ad0241f0) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:04.469450 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b723e5d-afd7-49d6-860d-115154dd15f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:04.470899 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b723e5d-afd7-49d6-860d-115154dd15f0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:04.483090 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 506/989] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:02:04 2022] GET /identity/v3/role_assignments?user.id=09c23d1c732c452a9597c647ad0241f0&scope.project.id=daf12585428b41a2b45bdbc85413a6c4&role.id=4564db14e2a543f18cb88a5bea5aada2 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:05.053276 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c4c7a0-ae2c-4244-96c2-f76e9e0ae07e None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:05.053665 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c4c7a0-ae2c-4244-96c2-f76e9e0ae07e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:05.053864 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c4c7a0-ae2c-4244-96c2-f76e9e0ae07e None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:05.055378 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 484/990] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:02:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:02:05.060616 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f669e4f-b856-4444-98fa-161f4fb48c33 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:05.060913 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f669e4f-b856-4444-98fa-161f4fb48c33 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:05.061222 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f669e4f-b856-4444-98fa-161f4fb48c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:05.078339 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3f669e4f-b856-4444-98fa-161f4fb48c33 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:02:05.079532 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3f669e4f-b856-4444-98fa-161f4fb48c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:02:05.162892 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 507/991] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:02:05 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:02:05.181735 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c4c7a0-ae2c-4244-96c2-f76e9e0ae07e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:05.282408 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:05.290875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c4c7a0-ae2c-4244-96c2-f76e9e0ae07e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:05.324456 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b5250c-41b4-4269-a6e6-41f59240efad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:05.325166 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5250c-41b4-4269-a6e6-41f59240efad None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:05.325445 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5250c-41b4-4269-a6e6-41f59240efad None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:05.325714 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b5250c-41b4-4269-a6e6-41f59240efad None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:05.326977 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b5250c-41b4-4269-a6e6-41f59240efad None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:05.328422 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b5250c-41b4-4269-a6e6-41f59240efad None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:05.340637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-14b5250c-41b4-4269-a6e6-41f59240efad None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 15:02:05.340637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:05.340637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 15:02:05.340637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 15:02:05.340637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 15:02:05.340637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 15:02:05.340637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 15:02:05.340637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:02:05.340637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 15:02:05.340637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:02:05.340637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 15:02:05.340637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:02:05.340637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:05.340637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:02:05.340637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:02:05.340637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:02:05.340637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:02:05.342401 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:02:05.342401 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:02:05.342401 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:02:05.342401 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:02:05.342401 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:02:05.342401 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:02:05.342401 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 15:02:05.342401 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 15:02:05.342401 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 15:02:05.342401 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 15:02:05.342401 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:02:05.342401 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:02:05.342401 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:02:05.342401 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:02:05.342401 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:02:05.342401 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:02:05.348345 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:02:05.348345 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:02:05.348345 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:02:05.348345 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:02:05.348345 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:02:05.348345 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:02:05.348345 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:02:05.348345 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:02:05.348345 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:02:05.348345 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:02:05.348345 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 15:02:05.348345 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:02:05.348345 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 27 15:02:05.348345 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:02:05.348345 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 485/992] 10.222.0.50 () {56 vars in 1126 bytes} [Fri May 27 15:02:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:02:05.350926 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f669e4f-b856-4444-98fa-161f4fb48c33 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:05.386446 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f15fb57-2360-4a50-8cc3-a23135d59048 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:05.386734 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f15fb57-2360-4a50-8cc3-a23135d59048 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:05.387043 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f15fb57-2360-4a50-8cc3-a23135d59048 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:05.388939 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 486/993] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:02:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 15:02:05.394630 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b4973-1e20-458a-a3fe-bf3a3ef27090 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:05.395041 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b4973-1e20-458a-a3fe-bf3a3ef27090 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:05.395349 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391b4973-1e20-458a-a3fe-bf3a3ef27090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:05.412964 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-391b4973-1e20-458a-a3fe-bf3a3ef27090 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:02:05.414141 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-391b4973-1e20-458a-a3fe-bf3a3ef27090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:02:05.449649 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:05.459293 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f669e4f-b856-4444-98fa-161f4fb48c33 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:05.485332 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 487/994] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:02:05 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 15:02:05.495847 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa44be4-6f2d-4696-8b42-1dedfc57e308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:05.496513 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa44be4-6f2d-4696-8b42-1dedfc57e308 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:05.496799 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa44be4-6f2d-4696-8b42-1dedfc57e308 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:05.497075 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa44be4-6f2d-4696-8b42-1dedfc57e308 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:05.498314 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa44be4-6f2d-4696-8b42-1dedfc57e308 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:05.498678 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa44be4-6f2d-4696-8b42-1dedfc57e308 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:05.500076 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa44be4-6f2d-4696-8b42-1dedfc57e308 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:05.505275 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 508/995] 10.222.0.50 () {56 vars in 1122 bytes} [Fri May 27 15:02:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:05.510068 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391b4973-1e20-458a-a3fe-bf3a3ef27090 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:05.511345 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa44be4-6f2d-4696-8b42-1dedfc57e308 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:05.605110 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:05.612857 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa44be4-6f2d-4696-8b42-1dedfc57e308 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:05.628450 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391b4973-1e20-458a-a3fe-bf3a3ef27090 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:05.651349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60082ae3-0453-4899-bd86-bf5fe7775eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:05.652015 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60082ae3-0453-4899-bd86-bf5fe7775eee None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:05.652302 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60082ae3-0453-4899-bd86-bf5fe7775eee None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:05.652579 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60082ae3-0453-4899-bd86-bf5fe7775eee None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:05.653281 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60082ae3-0453-4899-bd86-bf5fe7775eee None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:05.653587 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60082ae3-0453-4899-bd86-bf5fe7775eee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:05.655043 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60082ae3-0453-4899-bd86-bf5fe7775eee None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:05.660382 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 509/996] 10.222.0.50 () {56 vars in 1122 bytes} [Fri May 27 15:02:05 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:05.666115 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60082ae3-0453-4899-bd86-bf5fe7775eee None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:05.671277 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32227602-b310-425c-ab53-1cada6c0dadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:05.672269 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32227602-b310-425c-ab53-1cada6c0dadf None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:05.672709 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32227602-b310-425c-ab53-1cada6c0dadf None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:05.673144 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32227602-b310-425c-ab53-1cada6c0dadf None admin] PATH_INFO: `/v3/services/compute` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:05.673945 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32227602-b310-425c-ab53-1cada6c0dadf None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:05.675486 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32227602-b310-425c-ab53-1cada6c0dadf None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:05.689332 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-32227602-b310-425c-ab53-1cada6c0dadf None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 27 15:02:05.689332 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:05.689332 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 15:02:05.689332 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 15:02:05.689332 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 15:02:05.689332 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 15:02:05.689332 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 15:02:05.689332 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:02:05.689332 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 15:02:05.689332 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:02:05.689332 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 15:02:05.689332 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:02:05.689332 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:05.689332 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:02:05.689332 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:02:05.689332 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:02:05.689332 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:02:05.690603 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:02:05.690603 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:02:05.690603 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:02:05.690603 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:02:05.690603 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:02:05.690603 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:02:05.690603 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 15:02:05.690603 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 15:02:05.690603 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 15:02:05.690603 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 15:02:05.690603 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:02:05.690603 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:02:05.690603 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:02:05.690603 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:02:05.690603 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:02:05.690603 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:02:05.691661 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:02:05.691661 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:02:05.691661 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:02:05.691661 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:02:05.691661 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:02:05.691661 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:02:05.691661 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:02:05.691661 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:02:05.691661 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:02:05.691661 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:02:05.691661 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 15:02:05.691661 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:02:05.691661 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 15:02:05.691661 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:02:05.695446 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:05.695820 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 488/997] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:02:05 2022] GET /identity/v3/services/compute => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:02:05.696680 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32227602-b310-425c-ab53-1cada6c0dadf None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:05.779452 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60082ae3-0453-4899-bd86-bf5fe7775eee None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:05.806352 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32227602-b310-425c-ab53-1cada6c0dadf None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:05.815228 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c96f25-367e-49bf-97c0-056fd26a4149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:05.815959 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c96f25-367e-49bf-97c0-056fd26a4149 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:05.816160 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c96f25-367e-49bf-97c0-056fd26a4149 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:05.816429 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c96f25-367e-49bf-97c0-056fd26a4149 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:05.817159 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c96f25-367e-49bf-97c0-056fd26a4149 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:05.818522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c96f25-367e-49bf-97c0-056fd26a4149 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:05.834565 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 510/998] 10.222.0.50 () {60 vars in 1144 bytes} [Fri May 27 15:02:05 2022] POST /identity/v3/endpoints => generated 325 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:02:05.861433 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfb650b-84f3-40f5-9574-0a75357bfb40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:05.862565 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb650b-84f3-40f5-9574-0a75357bfb40 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:05.863131 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb650b-84f3-40f5-9574-0a75357bfb40 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:05.863606 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb650b-84f3-40f5-9574-0a75357bfb40 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:05.864582 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfb650b-84f3-40f5-9574-0a75357bfb40 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:05.865116 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfb650b-84f3-40f5-9574-0a75357bfb40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:05.866782 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfb650b-84f3-40f5-9574-0a75357bfb40 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:05.873763 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 489/999] 10.222.0.50 () {56 vars in 1112 bytes} [Fri May 27 15:02:05 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:05.879440 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c96f25-367e-49bf-97c0-056fd26a4149 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:05.969933 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:05.993100 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c96f25-367e-49bf-97c0-056fd26a4149 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:06.029890 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b28b049-702d-48b0-8965-6b604a1230ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:06.030563 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28b049-702d-48b0-8965-6b604a1230ca None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:06.030860 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28b049-702d-48b0-8965-6b604a1230ca None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:06.031151 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b28b049-702d-48b0-8965-6b604a1230ca None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:06.031853 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b28b049-702d-48b0-8965-6b604a1230ca None admin] RBAC: Adding query filter params (type=compute) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:06.032159 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b28b049-702d-48b0-8965-6b604a1230ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:06.033518 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b28b049-702d-48b0-8965-6b604a1230ca None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:06.038476 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 511/1000] 10.222.0.50 () {56 vars in 1112 bytes} [Fri May 27 15:02:05 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:06.044584 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfb650b-84f3-40f5-9574-0a75357bfb40 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:06.063145 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:06.155264 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfb650b-84f3-40f5-9574-0a75357bfb40 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:06.192591 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f9137-3cfe-4540-92fb-d44a6d204de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:06.193424 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f9137-3cfe-4540-92fb-d44a6d204de8 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:06.193709 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f9137-3cfe-4540-92fb-d44a6d204de8 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:06.193988 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f9137-3cfe-4540-92fb-d44a6d204de8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:06.194776 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157f9137-3cfe-4540-92fb-d44a6d204de8 None admin] RBAC: Adding query filter params (interface=public, service_id=9d7e80ae463f4b00bd3f7246914e244d, region_id=RegionOne) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:06.195124 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157f9137-3cfe-4540-92fb-d44a6d204de8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:06.196493 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157f9137-3cfe-4540-92fb-d44a6d204de8 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:06.203737 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 490/1001] 10.222.0.50 () {56 vars in 1251 bytes} [Fri May 27 15:02:06 2022] GET /identity/v3/endpoints?service_id=9d7e80ae463f4b00bd3f7246914e244d&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:06.209352 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b28b049-702d-48b0-8965-6b604a1230ca None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:06.314243 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b28b049-702d-48b0-8965-6b604a1230ca None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:06.348486 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f3da17-97e9-43e9-84c7-b4852c15d9a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:06.349179 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f3da17-97e9-43e9-84c7-b4852c15d9a8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:06.349467 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f3da17-97e9-43e9-84c7-b4852c15d9a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:06.349743 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f3da17-97e9-43e9-84c7-b4852c15d9a8 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:06.350422 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f3da17-97e9-43e9-84c7-b4852c15d9a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:06.351816 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f3da17-97e9-43e9-84c7-b4852c15d9a8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:06.358975 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 512/1002] 10.222.0.50 () {56 vars in 1087 bytes} [Fri May 27 15:02:06 2022] GET /identity/v3/services => generated 1727 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:02:06.733836 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642fb73-6d49-4aac-b3a4-a407d88b3300 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:06.734105 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642fb73-6d49-4aac-b3a4-a407d88b3300 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:06.734393 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d642fb73-6d49-4aac-b3a4-a407d88b3300 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:06.735842 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 491/1003] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:02:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:02:06.741091 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a357eb2a-0bba-4540-8b7d-d067a3bc7dd0 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:06.741389 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a357eb2a-0bba-4540-8b7d-d067a3bc7dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:06.741676 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a357eb2a-0bba-4540-8b7d-d067a3bc7dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:06.758350 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a357eb2a-0bba-4540-8b7d-d067a3bc7dd0 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:02:06.759552 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a357eb2a-0bba-4540-8b7d-d067a3bc7dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:02:06.763148 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:06.830543 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 513/1004] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:02:06 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:02:06.853489 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d642fb73-6d49-4aac-b3a4-a407d88b3300 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:06.932235 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:06.959115 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d642fb73-6d49-4aac-b3a4-a407d88b3300 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:06.995893 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee3831d-4162-4307-aa52-1309ecc6aa2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:06.996655 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3831d-4162-4307-aa52-1309ecc6aa2e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:06.996938 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3831d-4162-4307-aa52-1309ecc6aa2e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:06.997234 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3831d-4162-4307-aa52-1309ecc6aa2e None admin] PATH_INFO: `/v3/services/image` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:06.998413 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee3831d-4162-4307-aa52-1309ecc6aa2e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:06.999786 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee3831d-4162-4307-aa52-1309ecc6aa2e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:07.011564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-8ee3831d-4162-4307-aa52-1309ecc6aa2e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 27 15:02:07.011564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:07.011564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 15:02:07.011564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 15:02:07.011564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 15:02:07.011564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 15:02:07.011564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 15:02:07.011564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:02:07.011564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 15:02:07.011564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:02:07.011564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 15:02:07.011564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:02:07.011564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:07.011564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:02:07.011564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:02:07.011564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:02:07.011564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:02:07.012564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:02:07.012564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:02:07.012564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:02:07.012564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:02:07.012564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:02:07.012564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:02:07.012564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 15:02:07.012564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 15:02:07.012564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 15:02:07.012564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 15:02:07.012564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:02:07.012564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:02:07.012564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:02:07.012564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:02:07.012564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:02:07.012564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:02:07.019127 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:02:07.019127 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:02:07.019127 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:02:07.019127 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:02:07.019127 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:02:07.019127 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:02:07.019127 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:02:07.019127 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:02:07.019127 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:02:07.019127 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:02:07.019127 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 15:02:07.019127 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:02:07.019127 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 15:02:07.019127 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:02:07.019127 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 492/1005] 10.222.0.50 () {56 vars in 1105 bytes} [Fri May 27 15:02:06 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:02:07.021604 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a357eb2a-0bba-4540-8b7d-d067a3bc7dd0 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:07.125116 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a357eb2a-0bba-4540-8b7d-d067a3bc7dd0 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:07.167851 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da7e98d-fbd5-4682-8c12-78289dafe2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:07.168523 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7e98d-fbd5-4682-8c12-78289dafe2ed None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:07.168808 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7e98d-fbd5-4682-8c12-78289dafe2ed None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:07.169086 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7e98d-fbd5-4682-8c12-78289dafe2ed None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:07.169832 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da7e98d-fbd5-4682-8c12-78289dafe2ed None admin] RBAC: Adding query filter params (name=image) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:07.170140 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da7e98d-fbd5-4682-8c12-78289dafe2ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:07.171533 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da7e98d-fbd5-4682-8c12-78289dafe2ed None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:07.176681 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 514/1006] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:02:07 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:07.182212 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee3831d-4162-4307-aa52-1309ecc6aa2e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:07.282469 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee3831d-4162-4307-aa52-1309ecc6aa2e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:07.315270 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ab8de-0df8-4e55-8bfe-a1815917edb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:07.315923 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ab8de-0df8-4e55-8bfe-a1815917edb2 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:07.316205 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ab8de-0df8-4e55-8bfe-a1815917edb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:07.316485 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ab8de-0df8-4e55-8bfe-a1815917edb2 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:07.317178 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ab8de-0df8-4e55-8bfe-a1815917edb2 None admin] RBAC: Adding query filter params (type=image) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:07.317485 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ab8de-0df8-4e55-8bfe-a1815917edb2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:07.318861 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ab8de-0df8-4e55-8bfe-a1815917edb2 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:07.323433 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 493/1007] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:02:07 2022] GET /identity/v3/services?type=image => generated 122 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:08.095028 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7c7c6-f676-478b-bb8b-65ce7e8e0f74 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:08.095576 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7c7c6-f676-478b-bb8b-65ce7e8e0f74 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:08.095576 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7c7c6-f676-478b-bb8b-65ce7e8e0f74 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:08.097125 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 515/1008] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:02:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:02:08.102308 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbba48-72ef-444e-a55c-d99fcd982bfb None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:08.102605 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbba48-72ef-444e-a55c-d99fcd982bfb None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:08.102906 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbba48-72ef-444e-a55c-d99fcd982bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:08.109436 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:08.122195 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-87bbba48-72ef-444e-a55c-d99fcd982bfb None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:02:08.123984 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-87bbba48-72ef-444e-a55c-d99fcd982bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:02:08.198589 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 494/1009] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:02:08 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:02:08.223329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af7c7c6-f676-478b-bb8b-65ce7e8e0f74 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:08.227231 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:08.331948 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af7c7c6-f676-478b-bb8b-65ce7e8e0f74 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:08.366137 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf973a-8409-426e-ae43-96e383a74652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:08.366873 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf973a-8409-426e-ae43-96e383a74652 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:08.367171 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf973a-8409-426e-ae43-96e383a74652 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:08.367448 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf973a-8409-426e-ae43-96e383a74652 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:08.368630 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf973a-8409-426e-ae43-96e383a74652 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:08.370054 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bf973a-8409-426e-ae43-96e383a74652 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:08.371884 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 516/1010] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:02:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:08.378159 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bbba48-72ef-444e-a55c-d99fcd982bfb None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:08.484153 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bbba48-72ef-444e-a55c-d99fcd982bfb None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:08.522054 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37763d2-dc50-488f-90a2-7e7ad614bcf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:08.522711 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37763d2-dc50-488f-90a2-7e7ad614bcf4 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:08.523021 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37763d2-dc50-488f-90a2-7e7ad614bcf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:08.523296 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37763d2-dc50-488f-90a2-7e7ad614bcf4 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:08.524008 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37763d2-dc50-488f-90a2-7e7ad614bcf4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:08.525428 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37763d2-dc50-488f-90a2-7e7ad614bcf4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:08.528855 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:08.562880 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 495/1011] 10.222.0.50 () {60 vars in 1141 bytes} [Fri May 27 15:02:08 2022] POST /identity/v3/projects => generated 311 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:02:08.608982 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42b119-d30c-4758-86ea-25d119845976 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:08.609261 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42b119-d30c-4758-86ea-25d119845976 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:08.609566 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b42b119-d30c-4758-86ea-25d119845976 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:08.611039 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 517/1012] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:02:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:02:08.617250 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e96133-a853-468b-a670-2214c94bd069 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:08.617540 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e96133-a853-468b-a670-2214c94bd069 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:08.617741 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e96133-a853-468b-a670-2214c94bd069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:08.635222 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-85e96133-a853-468b-a670-2214c94bd069 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:02:08.636437 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-85e96133-a853-468b-a670-2214c94bd069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:02:08.710115 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 496/1013] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:02:08 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:02:08.734632 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b42b119-d30c-4758-86ea-25d119845976 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:08.812323 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:08.843147 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b42b119-d30c-4758-86ea-25d119845976 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:08.880202 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5346589-cc09-4084-bfc4-6fef8f16f548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:08.880915 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5346589-cc09-4084-bfc4-6fef8f16f548 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:08.881242 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5346589-cc09-4084-bfc4-6fef8f16f548 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:08.881529 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5346589-cc09-4084-bfc4-6fef8f16f548 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:08.882177 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5346589-cc09-4084-bfc4-6fef8f16f548 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:08.883564 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5346589-cc09-4084-bfc4-6fef8f16f548 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:08.895787 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-f5346589-cc09-4084-bfc4-6fef8f16f548 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 27 15:02:08.895787 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:08.895787 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 15:02:08.895787 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 15:02:08.895787 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 15:02:08.895787 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 15:02:08.895787 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 15:02:08.895787 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:02:08.895787 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 15:02:08.895787 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:02:08.895787 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 15:02:08.895787 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:02:08.895787 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:08.895787 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:02:08.895787 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:02:08.895787 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:02:08.895787 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:02:08.896892 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:02:08.896892 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:02:08.896892 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:02:08.896892 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:02:08.896892 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:02:08.896892 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:02:08.896892 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 15:02:08.896892 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 15:02:08.896892 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 15:02:08.896892 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 15:02:08.896892 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:02:08.896892 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:02:08.896892 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:02:08.896892 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:02:08.896892 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:02:08.896892 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:02:08.897788 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:02:08.897788 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:02:08.897788 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:02:08.897788 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:02:08.897788 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:02:08.897788 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:02:08.897788 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:02:08.897788 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:02:08.897788 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:02:08.897788 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:02:08.897788 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 15:02:08.897788 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:02:08.897788 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 27 15:02:08.897788 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:02:08.897788 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 518/1014] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:02:08 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:02:08.903052 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e96133-a853-468b-a670-2214c94bd069 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:09.004941 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e96133-a853-468b-a670-2214c94bd069 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:09.021978 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:09.041084 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f415cec-8dd9-46dd-ba98-af9de9fcd7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:09.041750 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f415cec-8dd9-46dd-ba98-af9de9fcd7e0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:09.042036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f415cec-8dd9-46dd-ba98-af9de9fcd7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:09.042313 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f415cec-8dd9-46dd-ba98-af9de9fcd7e0 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:09.043049 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f415cec-8dd9-46dd-ba98-af9de9fcd7e0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:09.043361 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f415cec-8dd9-46dd-ba98-af9de9fcd7e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:09.045272 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f415cec-8dd9-46dd-ba98-af9de9fcd7e0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:09.050403 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 497/1015] 10.222.0.50 () {56 vars in 1112 bytes} [Fri May 27 15:02:08 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:02:09.055861 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5346589-cc09-4084-bfc4-6fef8f16f548 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:09.175160 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5346589-cc09-4084-bfc4-6fef8f16f548 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:09.209244 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856e2ddc-0ca1-4c86-8eee-4f5e0083290c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:09.209911 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e2ddc-0ca1-4c86-8eee-4f5e0083290c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:09.210196 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e2ddc-0ca1-4c86-8eee-4f5e0083290c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:09.210473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e2ddc-0ca1-4c86-8eee-4f5e0083290c None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:09.211283 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856e2ddc-0ca1-4c86-8eee-4f5e0083290c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:09.211594 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856e2ddc-0ca1-4c86-8eee-4f5e0083290c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:09.212945 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856e2ddc-0ca1-4c86-8eee-4f5e0083290c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:09.215288 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:09.223188 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 519/1016] 10.222.0.50 () {56 vars in 1112 bytes} [Fri May 27 15:02:09 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:09.225316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f415cec-8dd9-46dd-ba98-af9de9fcd7e0 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:09.327875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f415cec-8dd9-46dd-ba98-af9de9fcd7e0 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:09.363914 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad9fc6c-c9da-4cf8-9e19-fd4b7083fbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:09.364745 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9fc6c-c9da-4cf8-9e19-fd4b7083fbf0 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:09.365035 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9fc6c-c9da-4cf8-9e19-fd4b7083fbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:09.365311 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9fc6c-c9da-4cf8-9e19-fd4b7083fbf0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:09.366081 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad9fc6c-c9da-4cf8-9e19-fd4b7083fbf0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:09.367678 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad9fc6c-c9da-4cf8-9e19-fd4b7083fbf0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:09.394881 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 498/1017] 10.222.0.50 () {60 vars in 1144 bytes} [Fri May 27 15:02:09 2022] POST /identity/v3/endpoints => generated 334 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:02:09.565657 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7329b6-7e92-4163-a8c5-807e1f4f27c4 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:09.566049 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7329b6-7e92-4163-a8c5-807e1f4f27c4 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:09.566228 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7329b6-7e92-4163-a8c5-807e1f4f27c4 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:09.567657 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 520/1018] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:02:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:02:09.572837 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c982e-9219-44a5-819b-79ab242dc9fe None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:09.573243 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c982e-9219-44a5-819b-79ab242dc9fe None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:09.573532 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c982e-9219-44a5-819b-79ab242dc9fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:09.591492 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a97c982e-9219-44a5-819b-79ab242dc9fe None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:02:09.593158 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a97c982e-9219-44a5-819b-79ab242dc9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:02:09.622665 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:09.670345 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 499/1019] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:02:09 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:02:09.694501 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7329b6-7e92-4163-a8c5-807e1f4f27c4 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:09.801734 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7329b6-7e92-4163-a8c5-807e1f4f27c4 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:09.835342 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e502f5a-9815-47bd-b800-2e24af6b71c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:09.835925 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e502f5a-9815-47bd-b800-2e24af6b71c8 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:09.836207 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e502f5a-9815-47bd-b800-2e24af6b71c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:09.836481 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e502f5a-9815-47bd-b800-2e24af6b71c8 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:09.837214 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e502f5a-9815-47bd-b800-2e24af6b71c8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:09.838821 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e502f5a-9815-47bd-b800-2e24af6b71c8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:09.854246 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 521/1020] 10.222.0.50 () {60 vars in 1141 bytes} [Fri May 27 15:02:09 2022] POST /identity/v3/services => generated 242 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:02:10.840375 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb16eb-8b45-4ab7-a868-aaec0b0d585d None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:10.840912 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb16eb-8b45-4ab7-a868-aaec0b0d585d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:10.840912 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb16eb-8b45-4ab7-a868-aaec0b0d585d None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:10.842321 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 500/1021] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:02:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:02:10.848183 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92044f5-3ae8-47ac-b593-00b2c583bd24 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:10.848620 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92044f5-3ae8-47ac-b593-00b2c583bd24 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:10.848823 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92044f5-3ae8-47ac-b593-00b2c583bd24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:10.865584 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c92044f5-3ae8-47ac-b593-00b2c583bd24 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:02:10.866757 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c92044f5-3ae8-47ac-b593-00b2c583bd24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:02:10.923091 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:10.942968 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 522/1022] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:02:10 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:02:10.967867 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb16eb-8b45-4ab7-a868-aaec0b0d585d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:11.072944 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effb16eb-8b45-4ab7-a868-aaec0b0d585d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:11.106508 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acde63d7-3050-4afa-903d-4fa7cddca023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:11.107780 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde63d7-3050-4afa-903d-4fa7cddca023 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:11.108060 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde63d7-3050-4afa-903d-4fa7cddca023 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:11.108335 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acde63d7-3050-4afa-903d-4fa7cddca023 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:11.109518 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acde63d7-3050-4afa-903d-4fa7cddca023 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:11.110987 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acde63d7-3050-4afa-903d-4fa7cddca023 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:11.112809 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 501/1023] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:02:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:11.118203 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92044f5-3ae8-47ac-b593-00b2c583bd24 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:11.225910 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92044f5-3ae8-47ac-b593-00b2c583bd24 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:11.260665 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef720174-ccde-41aa-bc4f-3b667de5088f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:11.261347 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef720174-ccde-41aa-bc4f-3b667de5088f None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:11.261632 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef720174-ccde-41aa-bc4f-3b667de5088f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:11.261911 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef720174-ccde-41aa-bc4f-3b667de5088f None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:11.262627 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef720174-ccde-41aa-bc4f-3b667de5088f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:11.264130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef720174-ccde-41aa-bc4f-3b667de5088f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:11.285342 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 523/1024] 10.222.0.50 () {60 vars in 1132 bytes} [Fri May 27 15:02:11 2022] POST /identity/v3/users => generated 285 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:02:11.723881 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c517291-02bb-400c-8c9a-efccfa6ca8fb None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:11.724550 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c517291-02bb-400c-8c9a-efccfa6ca8fb None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:11.725036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c517291-02bb-400c-8c9a-efccfa6ca8fb None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:11.726600 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 502/1025] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:02:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:02:11.732246 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712fcc5c-bded-4ef4-becf-b299fb562770 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:11.732616 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712fcc5c-bded-4ef4-becf-b299fb562770 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:11.732832 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712fcc5c-bded-4ef4-becf-b299fb562770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:11.750342 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-712fcc5c-bded-4ef4-becf-b299fb562770 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:02:11.751569 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-712fcc5c-bded-4ef4-becf-b299fb562770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:02:11.788225 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:11.826908 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 524/1026] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:02:11 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:02:11.858512 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c517291-02bb-400c-8c9a-efccfa6ca8fb None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:11.891060 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:11.971160 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c517291-02bb-400c-8c9a-efccfa6ca8fb None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:12.006119 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aeccb87-1f72-4c6a-8f47-ef56edb451c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:12.006876 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeccb87-1f72-4c6a-8f47-ef56edb451c4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:12.007155 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeccb87-1f72-4c6a-8f47-ef56edb451c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:12.007432 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aeccb87-1f72-4c6a-8f47-ef56edb451c4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:12.018048 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aeccb87-1f72-4c6a-8f47-ef56edb451c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:02:12.026211 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-6aeccb87-1f72-4c6a-8f47-ef56edb451c4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 15:02:12.026211 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:12.026211 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:02:12.026211 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:02:12.026211 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:02:12.026211 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:02:12.026211 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:02:12.026211 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:02:12.026211 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:02:12.026211 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:02:12.026211 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:02:12.026211 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:02:12.026211 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:02:12.026211 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:02:12.026211 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:02:12.026211 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:02:12.027458 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:02:12.027458 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:02:12.027458 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:02:12.027458 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:02:12.027458 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:02:12.027458 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:02:12.027458 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:02:12.027458 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:02:12.027458 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:02:12.027458 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:02:12.027458 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:02:12.027458 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:02:12.027458 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:02:12.027458 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:02:12.027458 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:02:12.027458 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:02:12.028329 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:02:12.028329 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:02:12.028329 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:02:12.028329 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:02:12.028329 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:02:12.028329 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:02:12.028329 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:02:12.028329 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:02:12.028329 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:02:12.028329 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:02:12.028329 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 15:02:12.028329 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:02:12.028329 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 503/1027] 10.222.0.50 () {56 vars in 1120 bytes} [Fri May 27 15:02:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:02:12.034823 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712fcc5c-bded-4ef4-becf-b299fb562770 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:12.108379 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604971f7-371f-4d78-aaee-ead62059c0e5 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:12.108700 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604971f7-371f-4d78-aaee-ead62059c0e5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:12.108941 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604971f7-371f-4d78-aaee-ead62059c0e5 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:12.110336 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 504/1028] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:02:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:02:12.115428 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c938d9-e4f7-47ed-8ca5-8a7ff554b8dd None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:12.115722 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c938d9-e4f7-47ed-8ca5-8a7ff554b8dd None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:12.116005 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c938d9-e4f7-47ed-8ca5-8a7ff554b8dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:12.133422 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-51c938d9-e4f7-47ed-8ca5-8a7ff554b8dd None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:02:12.134621 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-51c938d9-e4f7-47ed-8ca5-8a7ff554b8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:02:12.158369 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712fcc5c-bded-4ef4-becf-b299fb562770 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:12.201392 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3317bca4-4ef0-42de-8426-2f813426ccc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:12.202066 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3317bca4-4ef0-42de-8426-2f813426ccc6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:12.202343 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3317bca4-4ef0-42de-8426-2f813426ccc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:12.202614 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3317bca4-4ef0-42de-8426-2f813426ccc6 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:12.203387 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3317bca4-4ef0-42de-8426-2f813426ccc6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:12.203692 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3317bca4-4ef0-42de-8426-2f813426ccc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:12.205036 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3317bca4-4ef0-42de-8426-2f813426ccc6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:12.212772 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 505/1029] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:02:12 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 15:02:12.218553 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 525/1030] 10.222.0.50 () {56 vars in 1115 bytes} [Fri May 27 15:02:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:02:12.219827 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c938d9-e4f7-47ed-8ca5-8a7ff554b8dd None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:12.237330 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3317bca4-4ef0-42de-8426-2f813426ccc6 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:12.241862 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:12.280080 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:12.327707 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c938d9-e4f7-47ed-8ca5-8a7ff554b8dd None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:12.353516 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3317bca4-4ef0-42de-8426-2f813426ccc6 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:12.363343 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65ef50f-e6cd-4df3-8360-4e769e15f52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:12.364135 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ef50f-e6cd-4df3-8360-4e769e15f52a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:12.364416 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ef50f-e6cd-4df3-8360-4e769e15f52a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:12.364706 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65ef50f-e6cd-4df3-8360-4e769e15f52a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:12.376205 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65ef50f-e6cd-4df3-8360-4e769e15f52a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:12.377658 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65ef50f-e6cd-4df3-8360-4e769e15f52a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:12.389493 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beebb5f1-3368-47ba-821d-213369b7b325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:12.390271 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beebb5f1-3368-47ba-821d-213369b7b325 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:12.390551 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beebb5f1-3368-47ba-821d-213369b7b325 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:12.390831 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beebb5f1-3368-47ba-821d-213369b7b325 None admin] PATH_INFO: `/v3/services/image` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:12.391499 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beebb5f1-3368-47ba-821d-213369b7b325 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:12.392833 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beebb5f1-3368-47ba-821d-213369b7b325 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:12.396683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-b65ef50f-e6cd-4df3-8360-4e769e15f52a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:02:12.396683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:12.396683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:02:12.396683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:02:12.396683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:02:12.396683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:02:12.396683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:02:12.396683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:02:12.396683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:02:12.396683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:02:12.396683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:02:12.396683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:02:12.396683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:02:12.396683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:02:12.396683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:02:12.397551 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:02:12.397551 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:02:12.397551 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:02:12.397551 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:02:12.397551 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:02:12.397551 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:02:12.397551 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:02:12.397551 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:02:12.397551 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:02:12.397551 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:02:12.397551 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:02:12.397551 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:02:12.397551 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:02:12.397551 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:02:12.397551 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:02:12.397551 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:02:12.397551 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:02:12.398454 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:02:12.398454 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:02:12.398454 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:02:12.398454 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:02:12.402663 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 506/1031] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:02:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:02:12.403297 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65ef50f-e6cd-4df3-8360-4e769e15f52a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:12.404416 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-beebb5f1-3368-47ba-821d-213369b7b325 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 27 15:02:12.404416 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:12.404416 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 15:02:12.404416 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 15:02:12.404416 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 15:02:12.404416 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 15:02:12.404416 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 15:02:12.404416 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:02:12.404416 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 15:02:12.404416 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:02:12.404416 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 15:02:12.404416 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:02:12.404416 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:12.404416 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:02:12.404416 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:02:12.404416 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:02:12.404416 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:02:12.405380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:02:12.405380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:02:12.405380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:02:12.405380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:02:12.405380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:02:12.405380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:02:12.405380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 15:02:12.405380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 15:02:12.405380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 15:02:12.405380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 15:02:12.405380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:02:12.405380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:02:12.405380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:02:12.405380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:02:12.405380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:02:12.405380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:02:12.407957 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:02:12.407957 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:02:12.407957 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:02:12.407957 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:02:12.407957 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:02:12.407957 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:02:12.407957 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:02:12.407957 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:02:12.407957 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:02:12.407957 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:02:12.407957 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 15:02:12.407957 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:02:12.407957 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 15:02:12.407957 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:02:12.407957 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 526/1032] 10.222.0.50 () {56 vars in 1105 bytes} [Fri May 27 15:02:12 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:02:12.411685 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beebb5f1-3368-47ba-821d-213369b7b325 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:12.490436 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:12.525640 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65ef50f-e6cd-4df3-8360-4e769e15f52a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:12.526210 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:12.531628 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beebb5f1-3368-47ba-821d-213369b7b325 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:12.564280 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f71ad-1b11-4fa0-bb31-9c9d1df60f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:12.565212 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f71ad-1b11-4fa0-bb31-9c9d1df60f16 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:12.565599 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f71ad-1b11-4fa0-bb31-9c9d1df60f16 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:12.565882 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f71ad-1b11-4fa0-bb31-9c9d1df60f16 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:12.566771 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21f71ad-1b11-4fa0-bb31-9c9d1df60f16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:12.567113 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21f71ad-1b11-4fa0-bb31-9c9d1df60f16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:12.568518 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3406f4-5feb-4ba0-96aa-4970304514b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:12.568635 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21f71ad-1b11-4fa0-bb31-9c9d1df60f16 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:12.569178 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3406f4-5feb-4ba0-96aa-4970304514b6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:12.569460 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3406f4-5feb-4ba0-96aa-4970304514b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:12.569735 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3406f4-5feb-4ba0-96aa-4970304514b6 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:12.570431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3406f4-5feb-4ba0-96aa-4970304514b6 None admin] RBAC: Adding query filter params (name=image) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:12.570734 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3406f4-5feb-4ba0-96aa-4970304514b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:12.572140 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3406f4-5feb-4ba0-96aa-4970304514b6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:12.582543 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 527/1033] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:02:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:02:12.584879 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3406f4-5feb-4ba0-96aa-4970304514b6 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:12.591304 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 507/1034] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:02:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:02:12.592373 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f71ad-1b11-4fa0-bb31-9c9d1df60f16 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:12.694171 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3406f4-5feb-4ba0-96aa-4970304514b6 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:12.699890 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f71ad-1b11-4fa0-bb31-9c9d1df60f16 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:12.729812 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ba335-c240-432c-b1ef-08fb02421f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:12.730505 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ba335-c240-432c-b1ef-08fb02421f7f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:12.730788 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ba335-c240-432c-b1ef-08fb02421f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:12.731088 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ba335-c240-432c-b1ef-08fb02421f7f None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:12.731781 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79ba335-c240-432c-b1ef-08fb02421f7f None admin] RBAC: Adding query filter params (type=image) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:12.732623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79ba335-c240-432c-b1ef-08fb02421f7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:12.733974 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79ba335-c240-432c-b1ef-08fb02421f7f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:12.735389 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:12.739150 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 528/1035] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:02:12 2022] GET /identity/v3/services?type=image => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:02:12.744861 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ba335-c240-432c-b1ef-08fb02421f7f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:12.746219 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24433f2-1963-4ebd-83d3-6fed97555f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:12.747045 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24433f2-1963-4ebd-83d3-6fed97555f4c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:12.747335 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24433f2-1963-4ebd-83d3-6fed97555f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:12.748238 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24433f2-1963-4ebd-83d3-6fed97555f4c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:12.755978 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24433f2-1963-4ebd-83d3-6fed97555f4c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:12.757421 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24433f2-1963-4ebd-83d3-6fed97555f4c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:12.772827 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-e24433f2-1963-4ebd-83d3-6fed97555f4c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 15:02:12.772827 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:12.772827 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:02:12.772827 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:02:12.772827 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:02:12.772827 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:02:12.772827 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:02:12.772827 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:02:12.772827 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:02:12.772827 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:02:12.772827 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:02:12.772827 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:02:12.772827 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 15:02:12.772827 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 15:02:12.772827 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 15:02:12.772827 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 15:02:12.773784 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:02:12.773784 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:02:12.773784 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 15:02:12.773784 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:02:12.773784 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 15:02:12.773784 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:02:12.773784 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:02:12.773784 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:02:12.773784 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:02:12.773784 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:02:12.773784 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:02:12.773784 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:02:12.773784 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:02:12.773784 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:02:12.773784 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:02:12.773784 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:02:12.773784 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:02:12.773784 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:02:12.775379 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:02:12.775379 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:02:12.775379 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 15:02:12.775379 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 15:02:12.775379 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 15:02:12.775379 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 15:02:12.775379 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 15:02:12.775379 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 15:02:12.775379 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 15:02:12.775379 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:02:12.775379 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 508/1036] 10.222.0.50 () {56 vars in 1093 bytes} [Fri May 27 15:02:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:02:12.777675 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:12.780083 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24433f2-1963-4ebd-83d3-6fed97555f4c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:12.873554 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ba335-c240-432c-b1ef-08fb02421f7f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:12.905550 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24433f2-1963-4ebd-83d3-6fed97555f4c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:12.910197 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f47dd6-20fc-47e7-bf0f-46af7f949847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:12.910877 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f47dd6-20fc-47e7-bf0f-46af7f949847 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:12.911163 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f47dd6-20fc-47e7-bf0f-46af7f949847 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:12.911438 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f47dd6-20fc-47e7-bf0f-46af7f949847 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:12.912246 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f47dd6-20fc-47e7-bf0f-46af7f949847 None admin] RBAC: Adding query filter params (interface=public, service_id=bdde7a094e7b46528bb7a1d98ef27a7c, region_id=RegionOne) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:12.912559 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f47dd6-20fc-47e7-bf0f-46af7f949847 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:12.913923 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f47dd6-20fc-47e7-bf0f-46af7f949847 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:12.919028 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 529/1037] 10.222.0.50 () {56 vars in 1251 bytes} [Fri May 27 15:02:12 2022] GET /identity/v3/endpoints?service_id=bdde7a094e7b46528bb7a1d98ef27a7c&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:12.924928 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f47dd6-20fc-47e7-bf0f-46af7f949847 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:12.949578 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a35b1f-27fd-4e5f-a117-f3353935f38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:12.950206 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a35b1f-27fd-4e5f-a117-f3353935f38d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:12.950486 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a35b1f-27fd-4e5f-a117-f3353935f38d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:12.950944 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a35b1f-27fd-4e5f-a117-f3353935f38d None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:12.951762 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a35b1f-27fd-4e5f-a117-f3353935f38d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:12.952200 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a35b1f-27fd-4e5f-a117-f3353935f38d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:12.953480 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a35b1f-27fd-4e5f-a117-f3353935f38d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:12.976863 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 509/1038] 10.222.0.50 () {56 vars in 1133 bytes} [Fri May 27 15:02:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:12.978073 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a35b1f-27fd-4e5f-a117-f3353935f38d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:13.022648 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:13.031064 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:13.035881 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f47dd6-20fc-47e7-bf0f-46af7f949847 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:13.070210 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b21977-4153-436b-92ef-8a7086344e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:13.070917 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b21977-4153-436b-92ef-8a7086344e6a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:13.071203 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b21977-4153-436b-92ef-8a7086344e6a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:13.071475 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b21977-4153-436b-92ef-8a7086344e6a None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:13.072176 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b21977-4153-436b-92ef-8a7086344e6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:13.073526 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b21977-4153-436b-92ef-8a7086344e6a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:13.080136 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 530/1039] 10.222.0.50 () {56 vars in 1087 bytes} [Fri May 27 15:02:12 2022] GET /identity/v3/services => generated 1957 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:02:13.103066 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a35b1f-27fd-4e5f-a117-f3353935f38d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:13.144067 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c14af43-60a9-473e-a2c1-fb346d36be13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:13.144655 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14af43-60a9-473e-a2c1-fb346d36be13 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:13.144940 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14af43-60a9-473e-a2c1-fb346d36be13 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:13.145218 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c14af43-60a9-473e-a2c1-fb346d36be13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:13.225004 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c14af43-60a9-473e-a2c1-fb346d36be13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:13.226485 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c14af43-60a9-473e-a2c1-fb346d36be13 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:13.281811 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:13.336344 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 510/1040] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:02:12 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 15:02:13.343072 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b21977-4153-436b-92ef-8a7086344e6a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:13.443506 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b21977-4153-436b-92ef-8a7086344e6a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:13.477630 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a4672a-be49-4a5f-a1bc-da684f2e27de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:13.478426 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4672a-be49-4a5f-a1bc-da684f2e27de None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:13.478710 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4672a-be49-4a5f-a1bc-da684f2e27de None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:13.479147 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a4672a-be49-4a5f-a1bc-da684f2e27de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:13.489874 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a4672a-be49-4a5f-a1bc-da684f2e27de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:13.491360 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a4672a-be49-4a5f-a1bc-da684f2e27de None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:13.509420 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-92a4672a-be49-4a5f-a1bc-da684f2e27de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:02:13.509420 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:13.509420 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:02:13.509420 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:02:13.509420 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:02:13.509420 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:02:13.509420 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:02:13.509420 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:02:13.509420 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:02:13.509420 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:02:13.509420 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:02:13.509420 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:02:13.509420 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:02:13.509420 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:02:13.509420 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:02:13.510443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:02:13.510443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:02:13.510443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:02:13.510443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:02:13.510443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:02:13.510443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:02:13.510443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:02:13.510443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:02:13.510443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:02:13.510443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:02:13.510443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:02:13.510443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:02:13.510443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:02:13.510443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:02:13.510443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:02:13.510443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:02:13.510443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:02:13.511439 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:02:13.511439 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:02:13.511439 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:02:13.511439 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:02:13.514962 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 531/1041] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:02:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:02:13.517581 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c14af43-60a9-473e-a2c1-fb346d36be13 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:13.619446 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c14af43-60a9-473e-a2c1-fb346d36be13 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:13.653704 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbde1302-6975-4ff0-9317-1e61ce3ac48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:13.654950 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde1302-6975-4ff0-9317-1e61ce3ac48a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:13.655234 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde1302-6975-4ff0-9317-1e61ce3ac48a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:13.655508 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbde1302-6975-4ff0-9317-1e61ce3ac48a None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:13.656227 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbde1302-6975-4ff0-9317-1e61ce3ac48a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:13.656535 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbde1302-6975-4ff0-9317-1e61ce3ac48a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:13.657893 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbde1302-6975-4ff0-9317-1e61ce3ac48a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:13.669578 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 511/1042] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:02:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:13.675109 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a4672a-be49-4a5f-a1bc-da684f2e27de None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:13.684425 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:13.699419 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73cf15c-39f5-40aa-8713-df84c1973b05 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:13.699636 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73cf15c-39f5-40aa-8713-df84c1973b05 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:13.699909 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73cf15c-39f5-40aa-8713-df84c1973b05 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:13.702031 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 512/1043] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:02:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 15:02:13.706924 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0c352-712b-4188-8485-7f80b9909ee3 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:13.707207 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0c352-712b-4188-8485-7f80b9909ee3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:13.707488 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0c352-712b-4188-8485-7f80b9909ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:13.724570 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5fd0c352-712b-4188-8485-7f80b9909ee3 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:02:13.725772 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5fd0c352-712b-4188-8485-7f80b9909ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:02:13.728567 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:13.787789 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a4672a-be49-4a5f-a1bc-da684f2e27de None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:13.799238 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 513/1044] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:02:13 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 15:02:13.820705 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0c352-712b-4188-8485-7f80b9909ee3 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:13.824779 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d047b-5f11-4602-8d1d-1614801ef091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:13.825575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d047b-5f11-4602-8d1d-1614801ef091 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:13.826439 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d047b-5f11-4602-8d1d-1614801ef091 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:13.826720 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d047b-5f11-4602-8d1d-1614801ef091 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:13.834519 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6d047b-5f11-4602-8d1d-1614801ef091 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:13.836003 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6d047b-5f11-4602-8d1d-1614801ef091 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:13.852525 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-5b6d047b-5f11-4602-8d1d-1614801ef091 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:02:13.852525 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:13.852525 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:02:13.852525 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:02:13.852525 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:02:13.852525 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:02:13.852525 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:02:13.852525 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:02:13.852525 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:02:13.852525 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:02:13.852525 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:02:13.852525 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:02:13.852525 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 15:02:13.852525 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 15:02:13.852525 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 15:02:13.853512 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 15:02:13.853512 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:02:13.853512 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:02:13.853512 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:02:13.853512 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:02:13.853512 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:02:13.853512 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:02:13.853512 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:02:13.853512 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:02:13.853512 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:02:13.853512 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:02:13.853512 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:02:13.853512 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:02:13.853512 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:02:13.853512 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:02:13.853512 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:02:13.853512 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:02:13.854433 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 15:02:13.854433 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 15:02:13.854433 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 15:02:13.854433 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 15:02:13.854433 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 15:02:13.854433 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 15:02:13.854433 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:02:13.854433 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:02:13.854433 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 532/1045] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:02:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:02:13.859716 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d047b-5f11-4602-8d1d-1614801ef091 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:13.927108 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:13.931805 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0c352-712b-4188-8485-7f80b9909ee3 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:13.967281 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d047b-5f11-4602-8d1d-1614801ef091 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:13.968994 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae3a0ce-fc5a-41d7-9143-a62b09deb340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:13.970567 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3a0ce-fc5a-41d7-9143-a62b09deb340 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:13.970879 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3a0ce-fc5a-41d7-9143-a62b09deb340 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:13.971216 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae3a0ce-fc5a-41d7-9143-a62b09deb340 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:13.980489 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:13.984956 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae3a0ce-fc5a-41d7-9143-a62b09deb340 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:02:13.995599 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-7ae3a0ce-fc5a-41d7-9143-a62b09deb340 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:02:13.995599 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:13.995599 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:02:13.995599 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:02:13.995599 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:02:13.995599 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:02:13.995599 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:02:13.995599 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:02:13.995599 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:02:13.995599 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:02:13.995599 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:02:13.995599 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:02:13.995599 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:02:13.995599 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:02:13.995599 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:02:13.995599 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:02:13.996573 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:02:13.996573 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:02:13.996573 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:02:13.996573 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:02:13.996573 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:02:13.996573 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:02:13.996573 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:02:13.996573 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:02:13.996573 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:02:13.996573 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:02:13.996573 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:02:13.996573 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:02:13.996573 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:02:13.996573 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:02:13.996573 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:02:13.996573 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:02:14.002757 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:02:14.002757 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:02:14.002757 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:02:14.002757 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:02:14.002757 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:02:14.002757 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:02:14.002757 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:02:14.002757 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:02:14.002757 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:02:14.002757 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:02:14.002757 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:02:14.002757 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:02:14.002757 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 514/1046] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 15:02:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:02:14.003698 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6f7c39-9f20-4b03-aa7c-690e149573c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:14.003799 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae3a0ce-fc5a-41d7-9143-a62b09deb340 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:14.004066 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f7c39-9f20-4b03-aa7c-690e149573c7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:14.004258 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f7c39-9f20-4b03-aa7c-690e149573c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:14.004537 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f7c39-9f20-4b03-aa7c-690e149573c7 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:14.005330 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6f7c39-9f20-4b03-aa7c-690e149573c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:14.005639 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6f7c39-9f20-4b03-aa7c-690e149573c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:14.007343 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6f7c39-9f20-4b03-aa7c-690e149573c7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:14.019504 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 533/1047] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 15:02:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:02:14.025453 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6f7c39-9f20-4b03-aa7c-690e149573c7 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:14.117506 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae3a0ce-fc5a-41d7-9143-a62b09deb340 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:14.133364 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6f7c39-9f20-4b03-aa7c-690e149573c7 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:14.160366 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6420fd-28d3-44d1-bb4b-1f3d08069bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:14.161098 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6420fd-28d3-44d1-bb4b-1f3d08069bbc None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:14.161384 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6420fd-28d3-44d1-bb4b-1f3d08069bbc None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:14.161984 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6420fd-28d3-44d1-bb4b-1f3d08069bbc None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:14.162433 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6420fd-28d3-44d1-bb4b-1f3d08069bbc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:14.162731 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6420fd-28d3-44d1-bb4b-1f3d08069bbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:14.164137 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6420fd-28d3-44d1-bb4b-1f3d08069bbc None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:14.168086 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:14.173152 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 515/1048] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:02:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:14.178568 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6420fd-28d3-44d1-bb4b-1f3d08069bbc None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:14.181233 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7c4f62-40aa-4859-81fe-899d7061d8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:14.182176 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c4f62-40aa-4859-81fe-899d7061d8d1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:14.182458 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c4f62-40aa-4859-81fe-899d7061d8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:14.182737 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c4f62-40aa-4859-81fe-899d7061d8d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:14.183603 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7c4f62-40aa-4859-81fe-899d7061d8d1 None admin] RBAC: Adding query filter params (role.id=4564db14e2a543f18cb88a5bea5aada2, scope.project.id=daf12585428b41a2b45bdbc85413a6c4, user.id=451a9232f95e494683d81b6078a318d0) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:14.183912 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7c4f62-40aa-4859-81fe-899d7061d8d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:14.185362 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7c4f62-40aa-4859-81fe-899d7061d8d1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:14.198892 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 534/1049] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:02:14 2022] GET /identity/v3/role_assignments?user.id=451a9232f95e494683d81b6078a318d0&scope.project.id=daf12585428b41a2b45bdbc85413a6c4&role.id=4564db14e2a543f18cb88a5bea5aada2 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:02:14.257774 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:14.304081 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6420fd-28d3-44d1-bb4b-1f3d08069bbc None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:14.340989 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c212f72-7764-494e-8eb8-260e62c30acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:14.342654 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c212f72-7764-494e-8eb8-260e62c30acb None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:14.343186 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c212f72-7764-494e-8eb8-260e62c30acb None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:14.343691 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c212f72-7764-494e-8eb8-260e62c30acb None admin] PATH_INFO: `/v3/users/613920d770f541b6b96901bd56b268db` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:14.360162 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c212f72-7764-494e-8eb8-260e62c30acb None admin] RBAC: Authorizing `identity:get_user(user_id=613920d770f541b6b96901bd56b268db)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:14.361648 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c212f72-7764-494e-8eb8-260e62c30acb None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:14.368113 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 516/1050] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:02:14 2022] GET /identity/v3/users/613920d770f541b6b96901bd56b268db => generated 285 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:02:14.369229 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7c4f62-40aa-4859-81fe-899d7061d8d1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:14.473243 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7c4f62-40aa-4859-81fe-899d7061d8d1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:14.506680 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b92397-8e3c-4c70-84eb-3ded1b2783ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:14.507416 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b92397-8e3c-4c70-84eb-3ded1b2783ad None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:14.507703 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b92397-8e3c-4c70-84eb-3ded1b2783ad None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:14.508032 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b92397-8e3c-4c70-84eb-3ded1b2783ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:14.565497 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:14.577804 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b92397-8e3c-4c70-84eb-3ded1b2783ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:14.579304 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b92397-8e3c-4c70-84eb-3ded1b2783ad None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:14.677572 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 535/1051] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:02:14 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 15:02:14.683175 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c212f72-7764-494e-8eb8-260e62c30acb None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:14.786715 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c212f72-7764-494e-8eb8-260e62c30acb None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:14.807260 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:14.821824 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04af5ca0-2ca6-4971-9be5-11aece3aefae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:14.822684 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af5ca0-2ca6-4971-9be5-11aece3aefae None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:14.822996 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af5ca0-2ca6-4971-9be5-11aece3aefae None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:14.823275 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04af5ca0-2ca6-4971-9be5-11aece3aefae None admin] PATH_INFO: `/v3/projects/31ce7d9017dc4804b6a367025d34990e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:14.824444 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04af5ca0-2ca6-4971-9be5-11aece3aefae None admin] RBAC: Authorizing `identity:get_project(project_id=31ce7d9017dc4804b6a367025d34990e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:14.826408 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04af5ca0-2ca6-4971-9be5-11aece3aefae None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:14.833110 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 517/1052] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:02:14 2022] GET /identity/v3/projects/31ce7d9017dc4804b6a367025d34990e => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:14.834658 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b92397-8e3c-4c70-84eb-3ded1b2783ad None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:14.929305 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:14.940281 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b92397-8e3c-4c70-84eb-3ded1b2783ad None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:14.976346 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3b847e-0051-4bd3-8ec3-119c366d1e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:14.977012 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b847e-0051-4bd3-8ec3-119c366d1e1f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:14.977297 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b847e-0051-4bd3-8ec3-119c366d1e1f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:14.977575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b847e-0051-4bd3-8ec3-119c366d1e1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:14.978409 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3b847e-0051-4bd3-8ec3-119c366d1e1f None admin] RBAC: Adding query filter params (role.id=759d845eb6284e42931b4265af448ebc, scope.project.id=31ce7d9017dc4804b6a367025d34990e, user.id=613920d770f541b6b96901bd56b268db) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:14.978721 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3b847e-0051-4bd3-8ec3-119c366d1e1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:14.980171 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3b847e-0051-4bd3-8ec3-119c366d1e1f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:14.994897 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 536/1053] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:02:14 2022] GET /identity/v3/role_assignments?user.id=613920d770f541b6b96901bd56b268db&scope.project.id=31ce7d9017dc4804b6a367025d34990e&role.id=759d845eb6284e42931b4265af448ebc => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:15.332095 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cafae63-c555-4a0d-87b4-82bac373e6e4 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:15.332632 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cafae63-c555-4a0d-87b4-82bac373e6e4 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:15.332632 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cafae63-c555-4a0d-87b4-82bac373e6e4 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:15.334231 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 518/1054] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:02:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:02:15.339481 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0f23cf-0535-446b-b556-5d54db1d74a6 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:15.339894 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0f23cf-0535-446b-b556-5d54db1d74a6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:15.340058 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0f23cf-0535-446b-b556-5d54db1d74a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:15.358418 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6a0f23cf-0535-446b-b556-5d54db1d74a6 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:02:15.359979 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6a0f23cf-0535-446b-b556-5d54db1d74a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:02:15.429009 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 537/1055] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:02:15 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:02:15.452182 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cafae63-c555-4a0d-87b4-82bac373e6e4 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:15.557759 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cafae63-c555-4a0d-87b4-82bac373e6e4 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:15.591205 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96976834-c4a8-4a0b-88c9-ef391382557a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:15.593088 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96976834-c4a8-4a0b-88c9-ef391382557a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:15.593559 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96976834-c4a8-4a0b-88c9-ef391382557a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:15.594020 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96976834-c4a8-4a0b-88c9-ef391382557a None admin] PATH_INFO: `/v3/services/image` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:15.595174 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96976834-c4a8-4a0b-88c9-ef391382557a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:15.597552 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96976834-c4a8-4a0b-88c9-ef391382557a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:15.613629 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-96976834-c4a8-4a0b-88c9-ef391382557a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 27 15:02:15.613629 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:15.613629 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 15:02:15.613629 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 15:02:15.613629 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 15:02:15.613629 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 15:02:15.613629 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 15:02:15.613629 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:02:15.613629 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 15:02:15.613629 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:02:15.613629 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 15:02:15.613629 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:02:15.613629 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:15.613629 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:02:15.613629 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:02:15.613629 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:02:15.613629 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:02:15.614744 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:02:15.614744 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:02:15.614744 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:02:15.614744 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:02:15.614744 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:02:15.614744 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:02:15.614744 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 15:02:15.614744 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 15:02:15.614744 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 15:02:15.614744 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 15:02:15.614744 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:02:15.614744 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:02:15.614744 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:02:15.614744 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:02:15.614744 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:02:15.614744 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:02:15.615644 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:02:15.615644 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:02:15.615644 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:02:15.615644 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:02:15.615644 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:02:15.615644 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:02:15.615644 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:02:15.615644 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:02:15.615644 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:02:15.615644 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:02:15.615644 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 15:02:15.615644 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:02:15.615644 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 27 15:02:15.615644 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:02:15.615644 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 519/1056] 10.222.0.50 () {56 vars in 1105 bytes} [Fri May 27 15:02:15 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:02:15.623569 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0f23cf-0535-446b-b556-5d54db1d74a6 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:15.706375 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:15.723501 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0f23cf-0535-446b-b556-5d54db1d74a6 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:15.759229 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a2ac97-dc4c-4883-8bdc-7093ea0a26e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:15.759893 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2ac97-dc4c-4883-8bdc-7093ea0a26e9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:15.760178 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2ac97-dc4c-4883-8bdc-7093ea0a26e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:15.760455 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2ac97-dc4c-4883-8bdc-7093ea0a26e9 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:15.761160 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a2ac97-dc4c-4883-8bdc-7093ea0a26e9 None admin] RBAC: Adding query filter params (name=image) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:15.761466 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a2ac97-dc4c-4883-8bdc-7093ea0a26e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:15.762890 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a2ac97-dc4c-4883-8bdc-7093ea0a26e9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:15.772738 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 538/1057] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:02:15 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:15.773930 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96976834-c4a8-4a0b-88c9-ef391382557a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:15.828804 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:15.878809 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96976834-c4a8-4a0b-88c9-ef391382557a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:15.912226 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd72a904-bf10-42af-8efc-ed747bced384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:15.912893 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72a904-bf10-42af-8efc-ed747bced384 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:15.913181 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72a904-bf10-42af-8efc-ed747bced384 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:15.913461 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd72a904-bf10-42af-8efc-ed747bced384 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:15.914169 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd72a904-bf10-42af-8efc-ed747bced384 None admin] RBAC: Adding query filter params (type=image) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:15.914482 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd72a904-bf10-42af-8efc-ed747bced384 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:15.915883 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd72a904-bf10-42af-8efc-ed747bced384 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:15.927731 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 520/1058] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:02:15 2022] GET /identity/v3/services?type=image => generated 350 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:15.927886 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a2ac97-dc4c-4883-8bdc-7093ea0a26e9 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:16.030577 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a2ac97-dc4c-4883-8bdc-7093ea0a26e9 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:16.064983 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac53b55-a030-4653-a024-87e767e4a7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:16.065639 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac53b55-a030-4653-a024-87e767e4a7b0 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:16.065924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac53b55-a030-4653-a024-87e767e4a7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:16.066198 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac53b55-a030-4653-a024-87e767e4a7b0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:16.066965 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac53b55-a030-4653-a024-87e767e4a7b0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:16.068357 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac53b55-a030-4653-a024-87e767e4a7b0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:16.094884 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 539/1059] 10.222.0.50 () {60 vars in 1144 bytes} [Fri May 27 15:02:15 2022] POST /identity/v3/endpoints => generated 327 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:02:16.409629 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4242956-162b-4c2c-83a5-ace9ef7ea51d None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:16.410137 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4242956-162b-4c2c-83a5-ace9ef7ea51d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:16.410137 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4242956-162b-4c2c-83a5-ace9ef7ea51d None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:16.411522 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 521/1060] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:02:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:02:16.416547 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2fdd25-a933-418f-a171-5d178c192381 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:16.416950 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2fdd25-a933-418f-a171-5d178c192381 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:16.417140 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2fdd25-a933-418f-a171-5d178c192381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:16.423269 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:16.436288 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2e2fdd25-a933-418f-a171-5d178c192381 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:02:16.437454 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2e2fdd25-a933-418f-a171-5d178c192381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:02:16.511058 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 540/1061] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:02:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:02:16.533149 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4242956-162b-4c2c-83a5-ace9ef7ea51d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:16.643082 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4242956-162b-4c2c-83a5-ace9ef7ea51d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:16.659740 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:16.679151 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c3dae5-367d-4aa8-8467-9c3d2552a90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:16.679958 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c3dae5-367d-4aa8-8467-9c3d2552a90f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:16.680236 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c3dae5-367d-4aa8-8467-9c3d2552a90f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:16.680506 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c3dae5-367d-4aa8-8467-9c3d2552a90f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:16.691273 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c3dae5-367d-4aa8-8467-9c3d2552a90f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:02:16.699258 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-36c3dae5-367d-4aa8-8467-9c3d2552a90f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 15:02:16.699258 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:16.699258 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:02:16.699258 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:02:16.699258 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:02:16.699258 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:02:16.699258 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:02:16.699258 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:02:16.699258 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:02:16.699258 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:02:16.699258 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:02:16.699258 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:02:16.699258 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:02:16.699258 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:02:16.699258 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:02:16.699258 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:02:16.700191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:02:16.700191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:02:16.700191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:02:16.700191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:02:16.700191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:02:16.700191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:02:16.700191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:02:16.700191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:02:16.700191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:02:16.700191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:02:16.700191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:02:16.700191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:02:16.700191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:02:16.700191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:02:16.700191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:02:16.700191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:02:16.701433 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:02:16.701433 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:02:16.701433 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:02:16.701433 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:02:16.701433 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:02:16.701433 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:02:16.701433 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:02:16.701433 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:02:16.701433 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:02:16.701433 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:02:16.701433 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 15:02:16.701433 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:02:16.703191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 522/1062] 10.222.0.50 () {56 vars in 1120 bytes} [Fri May 27 15:02:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:02:16.706414 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2fdd25-a933-418f-a171-5d178c192381 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:16.809314 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2fdd25-a933-418f-a171-5d178c192381 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:16.839289 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:16.847266 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4444bf6b-7034-4d7b-9a2a-ac45bcfb91fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:16.847941 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444bf6b-7034-4d7b-9a2a-ac45bcfb91fe None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:16.848221 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444bf6b-7034-4d7b-9a2a-ac45bcfb91fe None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:16.848493 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444bf6b-7034-4d7b-9a2a-ac45bcfb91fe None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:16.849235 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4444bf6b-7034-4d7b-9a2a-ac45bcfb91fe None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:16.849544 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4444bf6b-7034-4d7b-9a2a-ac45bcfb91fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:16.850915 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4444bf6b-7034-4d7b-9a2a-ac45bcfb91fe None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:16.864401 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 541/1063] 10.222.0.50 () {56 vars in 1115 bytes} [Fri May 27 15:02:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:16.865356 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c3dae5-367d-4aa8-8467-9c3d2552a90f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:16.968157 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c3dae5-367d-4aa8-8467-9c3d2552a90f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:17.001742 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aef89d-f18d-40d6-989c-3e618780d5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:17.002544 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aef89d-f18d-40d6-989c-3e618780d5a7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:17.002823 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aef89d-f18d-40d6-989c-3e618780d5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:17.003121 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aef89d-f18d-40d6-989c-3e618780d5a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:17.016188 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aef89d-f18d-40d6-989c-3e618780d5a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:17.017683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aef89d-f18d-40d6-989c-3e618780d5a7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:17.036761 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-37aef89d-f18d-40d6-989c-3e618780d5a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:02:17.036761 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:17.036761 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:02:17.036761 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:02:17.036761 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:02:17.036761 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:02:17.036761 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:02:17.036761 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:02:17.036761 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:02:17.036761 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:02:17.036761 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:02:17.036761 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:02:17.036761 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:02:17.036761 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:02:17.036761 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:02:17.037773 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:02:17.037773 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:02:17.037773 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:02:17.037773 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:02:17.037773 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:02:17.037773 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:02:17.037773 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:02:17.037773 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:02:17.037773 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:02:17.037773 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:02:17.037773 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:02:17.037773 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:02:17.037773 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:02:17.037773 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:02:17.037773 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:02:17.037773 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:02:17.037773 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:02:17.038668 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:02:17.038668 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:02:17.038668 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:02:17.038668 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:02:17.038668 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 523/1064] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:02:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:02:17.045525 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4444bf6b-7034-4d7b-9a2a-ac45bcfb91fe None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:17.147073 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4444bf6b-7034-4d7b-9a2a-ac45bcfb91fe None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:17.185792 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e85c700-4cdb-4837-947d-6827e40d1955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:17.186611 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85c700-4cdb-4837-947d-6827e40d1955 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:17.186921 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85c700-4cdb-4837-947d-6827e40d1955 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:17.187198 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85c700-4cdb-4837-947d-6827e40d1955 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:17.187914 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e85c700-4cdb-4837-947d-6827e40d1955 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:17.188219 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e85c700-4cdb-4837-947d-6827e40d1955 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:17.189574 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e85c700-4cdb-4837-947d-6827e40d1955 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:17.202896 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 542/1065] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:02:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:17.204691 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fdff5e-deef-49d0-9a20-827e6c260f48 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:17.205147 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fdff5e-deef-49d0-9a20-827e6c260f48 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:17.205634 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fdff5e-deef-49d0-9a20-827e6c260f48 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:17.207266 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 524/1066] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:02:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:02:17.209810 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e85c700-4cdb-4837-947d-6827e40d1955 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:17.212410 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92a587-f06c-44ee-a001-5ab15b3a5fae None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:17.212700 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92a587-f06c-44ee-a001-5ab15b3a5fae None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:17.212984 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92a587-f06c-44ee-a001-5ab15b3a5fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:17.231234 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-cf92a587-f06c-44ee-a001-5ab15b3a5fae None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:02:17.233137 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cf92a587-f06c-44ee-a001-5ab15b3a5fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:02:17.257401 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:17.266924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:17.312830 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 525/1067] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:02:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:02:17.322790 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e85c700-4cdb-4837-947d-6827e40d1955 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:17.335068 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf92a587-f06c-44ee-a001-5ab15b3a5fae None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:17.361710 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcef24d-8bb7-48b6-8ace-1ce371513b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:17.362756 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcef24d-8bb7-48b6-8ace-1ce371513b29 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:17.363510 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcef24d-8bb7-48b6-8ace-1ce371513b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:17.363799 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcef24d-8bb7-48b6-8ace-1ce371513b29 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:17.373661 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcef24d-8bb7-48b6-8ace-1ce371513b29 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:17.375174 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcef24d-8bb7-48b6-8ace-1ce371513b29 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:17.394755 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-0fcef24d-8bb7-48b6-8ace-1ce371513b29 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 15:02:17.394755 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:17.394755 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:02:17.394755 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:02:17.394755 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:02:17.394755 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:02:17.394755 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:02:17.394755 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:02:17.394755 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:02:17.394755 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:02:17.394755 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:02:17.394755 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:02:17.394755 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 15:02:17.394755 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 15:02:17.394755 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 15:02:17.394755 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 15:02:17.395803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:02:17.395803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:02:17.395803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 15:02:17.395803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:02:17.395803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 15:02:17.395803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:02:17.395803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:02:17.395803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:02:17.395803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:02:17.395803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:02:17.395803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:02:17.395803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:02:17.395803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:02:17.395803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:02:17.395803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:02:17.395803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:02:17.395803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:02:17.395803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:02:17.396783 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:02:17.396783 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:02:17.396783 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 15:02:17.396783 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 15:02:17.396783 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 15:02:17.396783 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 15:02:17.396783 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 15:02:17.396783 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 15:02:17.396783 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 15:02:17.396783 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:02:17.398122 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 543/1068] 10.222.0.50 () {56 vars in 1093 bytes} [Fri May 27 15:02:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:02:17.402344 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcef24d-8bb7-48b6-8ace-1ce371513b29 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:17.442772 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf92a587-f06c-44ee-a001-5ab15b3a5fae None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:17.479164 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7658a1-18eb-4a06-9ffe-f6687ce28748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:17.479903 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7658a1-18eb-4a06-9ffe-f6687ce28748 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:17.480187 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7658a1-18eb-4a06-9ffe-f6687ce28748 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:17.480462 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7658a1-18eb-4a06-9ffe-f6687ce28748 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:17.491742 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7658a1-18eb-4a06-9ffe-f6687ce28748 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:02:17.499330 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-2b7658a1-18eb-4a06-9ffe-f6687ce28748 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:02:17.499330 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:17.499330 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:02:17.499330 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:02:17.499330 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:02:17.499330 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:02:17.499330 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:02:17.499330 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:02:17.499330 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:02:17.499330 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:02:17.499330 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:02:17.499330 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:02:17.499330 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:02:17.499330 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:02:17.499330 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:02:17.499330 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:02:17.500489 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:02:17.500489 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:02:17.500489 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:02:17.500489 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:02:17.500489 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:02:17.500489 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:02:17.500489 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:02:17.500489 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:02:17.500489 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:02:17.500489 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:02:17.500489 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:02:17.500489 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:02:17.500489 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:02:17.500489 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:02:17.500489 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:02:17.500489 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:02:17.501455 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:02:17.501455 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:02:17.501455 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:02:17.501455 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:02:17.501455 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:02:17.501455 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:02:17.501455 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:02:17.501455 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:02:17.501455 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:02:17.501455 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:02:17.501455 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:02:17.501455 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:02:17.501455 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 526/1069] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 15:02:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:02:17.506249 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7658a1-18eb-4a06-9ffe-f6687ce28748 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:17.508341 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcef24d-8bb7-48b6-8ace-1ce371513b29 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:17.522686 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:17.527059 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:17.548497 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e911223-be22-4425-b422-9b5d74044fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:17.549162 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e911223-be22-4425-b422-9b5d74044fe3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:17.549446 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e911223-be22-4425-b422-9b5d74044fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:17.549722 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e911223-be22-4425-b422-9b5d74044fe3 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:17.550582 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e911223-be22-4425-b422-9b5d74044fe3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:17.550919 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e911223-be22-4425-b422-9b5d74044fe3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:17.552333 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e911223-be22-4425-b422-9b5d74044fe3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:17.574108 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 544/1070] 10.222.0.50 () {56 vars in 1133 bytes} [Fri May 27 15:02:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:02:17.578479 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e911223-be22-4425-b422-9b5d74044fe3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:17.619529 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7658a1-18eb-4a06-9ffe-f6687ce28748 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:17.656578 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bba7ab-9837-43ce-bccc-bf6953f951b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:17.657555 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bba7ab-9837-43ce-bccc-bf6953f951b2 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:17.657848 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bba7ab-9837-43ce-bccc-bf6953f951b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:17.658127 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bba7ab-9837-43ce-bccc-bf6953f951b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:17.658891 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bba7ab-9837-43ce-bccc-bf6953f951b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:17.659202 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bba7ab-9837-43ce-bccc-bf6953f951b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:17.660567 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bba7ab-9837-43ce-bccc-bf6953f951b2 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:17.669191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 527/1071] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:02:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:02:17.674757 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bba7ab-9837-43ce-bccc-bf6953f951b2 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:17.679332 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e911223-be22-4425-b422-9b5d74044fe3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:17.715659 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e651c6-a61c-4777-84a4-00419a3973f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:17.716517 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e651c6-a61c-4777-84a4-00419a3973f7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:17.716785 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e651c6-a61c-4777-84a4-00419a3973f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:17.717128 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e651c6-a61c-4777-84a4-00419a3973f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:17.727342 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e651c6-a61c-4777-84a4-00419a3973f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:17.728781 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e651c6-a61c-4777-84a4-00419a3973f7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:17.746488 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-b3e651c6-a61c-4777-84a4-00419a3973f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:02:17.746488 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:17.746488 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:02:17.746488 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:02:17.746488 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:02:17.746488 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:02:17.746488 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:02:17.746488 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:02:17.746488 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:02:17.746488 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:02:17.746488 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:02:17.746488 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:02:17.746488 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:02:17.746488 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:02:17.746488 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:02:17.752944 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:02:17.752944 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:02:17.752944 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:02:17.752944 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:02:17.752944 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:02:17.752944 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:02:17.752944 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:02:17.752944 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:02:17.752944 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:02:17.752944 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:02:17.752944 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:02:17.752944 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:02:17.752944 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:02:17.752944 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:02:17.752944 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:02:17.752944 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:02:17.752944 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:02:17.754133 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:02:17.754133 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:02:17.754133 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:02:17.754133 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:02:17.754133 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 545/1072] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:02:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:02:17.754133 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e651c6-a61c-4777-84a4-00419a3973f7 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:17.761141 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:17.780696 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:17.783463 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bba7ab-9837-43ce-bccc-bf6953f951b2 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:17.819990 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a860df-3d8d-414f-b6ed-8bdce9a100d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:17.821173 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a860df-3d8d-414f-b6ed-8bdce9a100d7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:17.821535 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a860df-3d8d-414f-b6ed-8bdce9a100d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:17.821821 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a860df-3d8d-414f-b6ed-8bdce9a100d7 None admin] PATH_INFO: `/v3/users/613920d770f541b6b96901bd56b268db` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:17.823074 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a860df-3d8d-414f-b6ed-8bdce9a100d7 None admin] RBAC: Authorizing `identity:get_user(user_id=613920d770f541b6b96901bd56b268db)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:17.824516 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a860df-3d8d-414f-b6ed-8bdce9a100d7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:17.830972 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 528/1073] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:02:17 2022] GET /identity/v3/users/613920d770f541b6b96901bd56b268db => generated 285 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:02:17.831872 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a860df-3d8d-414f-b6ed-8bdce9a100d7 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:17.873237 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e651c6-a61c-4777-84a4-00419a3973f7 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:17.910666 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b300922-100a-4df6-a18a-6891bd2ab298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:17.911507 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b300922-100a-4df6-a18a-6891bd2ab298 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:17.911792 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b300922-100a-4df6-a18a-6891bd2ab298 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:17.912083 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b300922-100a-4df6-a18a-6891bd2ab298 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:17.912783 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b300922-100a-4df6-a18a-6891bd2ab298 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:17.913090 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b300922-100a-4df6-a18a-6891bd2ab298 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:17.915060 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b300922-100a-4df6-a18a-6891bd2ab298 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:17.927219 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 546/1074] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:02:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:02:17.932498 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b300922-100a-4df6-a18a-6891bd2ab298 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:17.947279 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a860df-3d8d-414f-b6ed-8bdce9a100d7 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:17.983340 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0963f3fc-f855-463c-b28f-bbefa67d086d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:17.984207 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0963f3fc-f855-463c-b28f-bbefa67d086d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:17.984489 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0963f3fc-f855-463c-b28f-bbefa67d086d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:17.984764 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0963f3fc-f855-463c-b28f-bbefa67d086d None admin] PATH_INFO: `/v3/projects/31ce7d9017dc4804b6a367025d34990e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:17.985941 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0963f3fc-f855-463c-b28f-bbefa67d086d None admin] RBAC: Authorizing `identity:get_project(project_id=31ce7d9017dc4804b6a367025d34990e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:17.987425 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0963f3fc-f855-463c-b28f-bbefa67d086d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:17.993933 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 529/1075] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:02:17 2022] GET /identity/v3/projects/31ce7d9017dc4804b6a367025d34990e => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:02:17.994830 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0963f3fc-f855-463c-b28f-bbefa67d086d None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:18.015954 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:18.026162 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:18.050640 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b300922-100a-4df6-a18a-6891bd2ab298 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:18.086192 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c949eb5-59d7-4aee-b534-90985bb5851e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:18.087022 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c949eb5-59d7-4aee-b534-90985bb5851e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:18.087304 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c949eb5-59d7-4aee-b534-90985bb5851e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:18.087579 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c949eb5-59d7-4aee-b534-90985bb5851e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:18.094627 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c949eb5-59d7-4aee-b534-90985bb5851e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:18.096162 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c949eb5-59d7-4aee-b534-90985bb5851e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:18.109261 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0963f3fc-f855-463c-b28f-bbefa67d086d None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:18.110967 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-1c949eb5-59d7-4aee-b534-90985bb5851e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:02:18.110967 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:18.110967 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:02:18.110967 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:02:18.110967 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:02:18.110967 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:02:18.110967 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:02:18.110967 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:02:18.110967 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:02:18.110967 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:02:18.110967 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:02:18.110967 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:02:18.110967 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 15:02:18.110967 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 15:02:18.110967 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 15:02:18.111879 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 15:02:18.111879 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:02:18.111879 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:02:18.111879 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:02:18.111879 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:02:18.111879 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:02:18.111879 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:02:18.111879 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:02:18.111879 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:02:18.111879 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:02:18.111879 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:02:18.111879 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:02:18.111879 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:02:18.111879 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:02:18.111879 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:02:18.111879 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:02:18.111879 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:02:18.112806 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 15:02:18.112806 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 15:02:18.112806 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 15:02:18.112806 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 15:02:18.112806 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 15:02:18.112806 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 15:02:18.112806 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:02:18.112806 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:02:18.112806 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 547/1076] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:02:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:02:18.117734 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c949eb5-59d7-4aee-b534-90985bb5851e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:18.153156 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57de970-bea9-4fc6-96ce-38406e4b3923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:18.154146 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57de970-bea9-4fc6-96ce-38406e4b3923 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:18.157291 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57de970-bea9-4fc6-96ce-38406e4b3923 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:18.157291 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57de970-bea9-4fc6-96ce-38406e4b3923 None admin] PATH_INFO: `/v3/projects/31ce7d9017dc4804b6a367025d34990e/users/613920d770f541b6b96901bd56b268db/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:18.158899 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57de970-bea9-4fc6-96ce-38406e4b3923 None admin] RBAC: Authorizing `identity:create_grant(project_id=31ce7d9017dc4804b6a367025d34990e, user_id=613920d770f541b6b96901bd56b268db, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:18.160356 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57de970-bea9-4fc6-96ce-38406e4b3923 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:18.186890 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 530/1077] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 15:02:17 2022] PUT /identity/v3/projects/31ce7d9017dc4804b6a367025d34990e/users/613920d770f541b6b96901bd56b268db/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:02:18.235210 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c949eb5-59d7-4aee-b534-90985bb5851e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:18.272763 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd01467e-7329-49d9-9780-7e35c2283630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:18.273503 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01467e-7329-49d9-9780-7e35c2283630 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:18.273792 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01467e-7329-49d9-9780-7e35c2283630 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:18.274067 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01467e-7329-49d9-9780-7e35c2283630 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:18.274895 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd01467e-7329-49d9-9780-7e35c2283630 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:18.275207 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd01467e-7329-49d9-9780-7e35c2283630 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:18.276747 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd01467e-7329-49d9-9780-7e35c2283630 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:18.297320 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 548/1078] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 15:02:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:02:18.298172 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57de970-bea9-4fc6-96ce-38406e4b3923 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:18.388726 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:18.392758 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cede0f-93e2-4e59-933a-85a7b43ea603 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:18.393080 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cede0f-93e2-4e59-933a-85a7b43ea603 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:18.393366 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cede0f-93e2-4e59-933a-85a7b43ea603 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:18.394749 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 549/1079] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:02:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:02:18.400092 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aaff96-ef19-4261-bafc-8d370352630a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:18.400483 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aaff96-ef19-4261-bafc-8d370352630a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:18.400835 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aaff96-ef19-4261-bafc-8d370352630a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:18.407205 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:18.415338 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57de970-bea9-4fc6-96ce-38406e4b3923 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:18.424684 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-45aaff96-ef19-4261-bafc-8d370352630a None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:02:18.426729 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-45aaff96-ef19-4261-bafc-8d370352630a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:02:18.452253 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e9a0c2-ab1f-47e9-8597-c8e6e01f1113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:18.453147 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e9a0c2-ab1f-47e9-8597-c8e6e01f1113 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:18.453433 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e9a0c2-ab1f-47e9-8597-c8e6e01f1113 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:18.453711 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e9a0c2-ab1f-47e9-8597-c8e6e01f1113 None admin] PATH_INFO: `/v3/projects/daf12585428b41a2b45bdbc85413a6c4/users/451a9232f95e494683d81b6078a318d0/roles/4564db14e2a543f18cb88a5bea5aada2` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:18.455936 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e9a0c2-ab1f-47e9-8597-c8e6e01f1113 None admin] RBAC: Authorizing `identity:create_grant(project_id=daf12585428b41a2b45bdbc85413a6c4, user_id=451a9232f95e494683d81b6078a318d0, role_id=4564db14e2a543f18cb88a5bea5aada2)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:18.457384 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e9a0c2-ab1f-47e9-8597-c8e6e01f1113 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:18.473499 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 531/1080] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 15:02:18 2022] PUT /identity/v3/projects/daf12585428b41a2b45bdbc85413a6c4/users/451a9232f95e494683d81b6078a318d0/roles/4564db14e2a543f18cb88a5bea5aada2 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:02:18.511627 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 550/1081] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:02:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:02:18.534898 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e9a0c2-ab1f-47e9-8597-c8e6e01f1113 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:18.641655 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e9a0c2-ab1f-47e9-8597-c8e6e01f1113 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:18.678244 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084e40d8-9608-4542-aac9-6b4d214fffcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:18.678936 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e40d8-9608-4542-aac9-6b4d214fffcb None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:18.679220 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e40d8-9608-4542-aac9-6b4d214fffcb None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:18.679497 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084e40d8-9608-4542-aac9-6b4d214fffcb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:18.683237 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:18.753306 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084e40d8-9608-4542-aac9-6b4d214fffcb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:18.754970 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084e40d8-9608-4542-aac9-6b4d214fffcb None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:18.864021 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 532/1082] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:02:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 15:02:18.865642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aaff96-ef19-4261-bafc-8d370352630a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:18.975106 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aaff96-ef19-4261-bafc-8d370352630a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:19.006649 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:19.012930 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab5a8c-8038-4f8e-8653-e8083fe01771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:19.013853 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab5a8c-8038-4f8e-8653-e8083fe01771 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:19.014209 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab5a8c-8038-4f8e-8653-e8083fe01771 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:19.014551 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab5a8c-8038-4f8e-8653-e8083fe01771 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:19.025131 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ab5a8c-8038-4f8e-8653-e8083fe01771 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:19.026651 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ab5a8c-8038-4f8e-8653-e8083fe01771 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:19.045599 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-39ab5a8c-8038-4f8e-8653-e8083fe01771 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:02:19.045599 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:19.045599 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:02:19.045599 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:02:19.045599 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:02:19.045599 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:02:19.045599 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:02:19.045599 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:02:19.045599 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:02:19.045599 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:02:19.045599 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:02:19.045599 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:02:19.045599 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:02:19.045599 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:02:19.045599 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:02:19.046727 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:02:19.046727 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:02:19.046727 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:02:19.046727 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:02:19.046727 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:02:19.046727 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:02:19.046727 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:02:19.046727 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:02:19.046727 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:02:19.046727 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:02:19.046727 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:02:19.046727 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:02:19.046727 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:02:19.046727 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:02:19.046727 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:02:19.046727 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:02:19.046727 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:02:19.047907 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:02:19.047907 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:02:19.047907 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:02:19.047907 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:02:19.048885 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 551/1083] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:02:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:02:19.054503 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084e40d8-9608-4542-aac9-6b4d214fffcb None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:19.088275 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:19.159931 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084e40d8-9608-4542-aac9-6b4d214fffcb None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:19.199952 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af39bbcd-2603-4c0b-a865-965255084cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:19.200765 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af39bbcd-2603-4c0b-a865-965255084cb0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:19.201160 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af39bbcd-2603-4c0b-a865-965255084cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:19.201558 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af39bbcd-2603-4c0b-a865-965255084cb0 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:19.202291 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af39bbcd-2603-4c0b-a865-965255084cb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:19.202601 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af39bbcd-2603-4c0b-a865-965255084cb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:19.204013 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af39bbcd-2603-4c0b-a865-965255084cb0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:19.222880 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 533/1084] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:02:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:20.441327 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17272d29-1715-441e-a9e9-1b7078c68715 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:20.441877 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17272d29-1715-441e-a9e9-1b7078c68715 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:20.441877 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17272d29-1715-441e-a9e9-1b7078c68715 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:20.443431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 552/1085] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:02:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:02:20.448795 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d18dde-a0e1-41b7-9ecd-2abe4520ade9 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:20.449236 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d18dde-a0e1-41b7-9ecd-2abe4520ade9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:20.449368 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d18dde-a0e1-41b7-9ecd-2abe4520ade9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:20.466632 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-76d18dde-a0e1-41b7-9ecd-2abe4520ade9 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:02:20.467923 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-76d18dde-a0e1-41b7-9ecd-2abe4520ade9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:02:20.533442 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 534/1086] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:02:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:02:20.557549 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17272d29-1715-441e-a9e9-1b7078c68715 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:20.672407 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17272d29-1715-441e-a9e9-1b7078c68715 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:20.704841 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103f77bb-7883-4839-ab19-f5102f27993d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:20.705561 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f77bb-7883-4839-ab19-f5102f27993d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:20.705850 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f77bb-7883-4839-ab19-f5102f27993d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:20.706129 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f77bb-7883-4839-ab19-f5102f27993d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:20.716790 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103f77bb-7883-4839-ab19-f5102f27993d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:02:20.722614 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8a9e0-46b8-48c3-9f83-83cb3ccf1a9a None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:20.722935 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8a9e0-46b8-48c3-9f83-83cb3ccf1a9a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:20.723216 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8a9e0-46b8-48c3-9f83-83cb3ccf1a9a None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:20.724584 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 535/1087] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:02:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 15:02:20.724724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-103f77bb-7883-4839-ab19-f5102f27993d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:02:20.724724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:20.724724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:02:20.724724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:02:20.724724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:02:20.724724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:02:20.724724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:02:20.724724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:02:20.724724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:02:20.724724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:02:20.724724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:02:20.724724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:02:20.724724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:02:20.724724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:02:20.724724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:02:20.724724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:02:20.725719 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:02:20.725719 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:02:20.725719 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:02:20.725719 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:02:20.725719 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:02:20.725719 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:02:20.725719 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:02:20.725719 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:02:20.725719 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:02:20.725719 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:02:20.725719 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:02:20.725719 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:02:20.725719 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:02:20.725719 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:02:20.725719 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:02:20.725719 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:02:20.726643 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:02:20.726643 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:02:20.726643 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:02:20.726643 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:02:20.726643 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:02:20.726643 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:02:20.726643 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:02:20.726643 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:02:20.726643 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:02:20.726643 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:02:20.726643 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:02:20.726643 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:02:20.726643 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 553/1088] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 15:02:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:02:20.729710 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e61db-e993-4605-b0a1-aebe0fbb550b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:20.730002 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e61db-e993-4605-b0a1-aebe0fbb550b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:20.730283 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66e61db-e993-4605-b0a1-aebe0fbb550b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:20.731572 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103f77bb-7883-4839-ab19-f5102f27993d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:20.747810 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f66e61db-e993-4605-b0a1-aebe0fbb550b None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:02:20.748995 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f66e61db-e993-4605-b0a1-aebe0fbb550b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:02:20.768571 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:20.776527 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:20.816008 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 536/1089] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:02:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 15:02:20.832558 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103f77bb-7883-4839-ab19-f5102f27993d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:20.839714 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e61db-e993-4605-b0a1-aebe0fbb550b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:20.866636 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea8ecd-b8d3-4c41-b79c-2cc2256cd84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:20.867344 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea8ecd-b8d3-4c41-b79c-2cc2256cd84b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:20.867630 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea8ecd-b8d3-4c41-b79c-2cc2256cd84b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:20.867907 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cea8ecd-b8d3-4c41-b79c-2cc2256cd84b None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:20.868649 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea8ecd-b8d3-4c41-b79c-2cc2256cd84b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:20.868953 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea8ecd-b8d3-4c41-b79c-2cc2256cd84b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:20.870304 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cea8ecd-b8d3-4c41-b79c-2cc2256cd84b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:20.878669 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 554/1090] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:02:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:02:20.884536 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea8ecd-b8d3-4c41-b79c-2cc2256cd84b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:20.950291 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66e61db-e993-4605-b0a1-aebe0fbb550b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:20.986430 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7d8fc9-5a43-442b-b154-0daee52d1e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:20.987184 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d8fc9-5a43-442b-b154-0daee52d1e62 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:20.987472 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d8fc9-5a43-442b-b154-0daee52d1e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:20.987744 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d8fc9-5a43-442b-b154-0daee52d1e62 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:20.991081 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cea8ecd-b8d3-4c41-b79c-2cc2256cd84b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:20.997877 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7d8fc9-5a43-442b-b154-0daee52d1e62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:02:20.999045 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:21.006268 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-5f7d8fc9-5a43-442b-b154-0daee52d1e62 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 15:02:21.006268 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:21.006268 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:02:21.006268 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:02:21.006268 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:02:21.006268 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:02:21.006268 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:02:21.006268 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:02:21.006268 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:02:21.006268 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:02:21.006268 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:02:21.006268 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:02:21.006268 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:02:21.006268 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:02:21.006268 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:02:21.006268 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:02:21.007299 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:02:21.007299 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:02:21.007299 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:02:21.007299 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:02:21.007299 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:02:21.007299 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:02:21.007299 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:02:21.007299 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:02:21.007299 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:02:21.007299 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:02:21.007299 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:02:21.007299 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:02:21.007299 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:02:21.007299 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:02:21.007299 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:02:21.007299 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:02:21.008715 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:02:21.008715 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:02:21.008715 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:02:21.008715 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:02:21.008715 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:02:21.008715 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:02:21.008715 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:02:21.008715 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:02:21.008715 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:02:21.008715 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:02:21.008715 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 27 15:02:21.008715 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:02:21.008715 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 537/1091] 10.222.0.50 () {56 vars in 1120 bytes} [Fri May 27 15:02:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:02:21.013958 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7d8fc9-5a43-442b-b154-0daee52d1e62 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:21.025390 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189743d9-8329-46cd-9cee-b92f073b1a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:21.026255 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189743d9-8329-46cd-9cee-b92f073b1a26 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:21.026534 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189743d9-8329-46cd-9cee-b92f073b1a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:21.026804 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189743d9-8329-46cd-9cee-b92f073b1a26 None admin] PATH_INFO: `/v3/users/613920d770f541b6b96901bd56b268db` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:21.028014 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189743d9-8329-46cd-9cee-b92f073b1a26 None admin] RBAC: Authorizing `identity:get_user(user_id=613920d770f541b6b96901bd56b268db)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:21.030042 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189743d9-8329-46cd-9cee-b92f073b1a26 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:21.031698 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 555/1092] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:02:20 2022] GET /identity/v3/users/613920d770f541b6b96901bd56b268db => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:02:21.033215 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:21.037354 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189743d9-8329-46cd-9cee-b92f073b1a26 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:21.117607 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7d8fc9-5a43-442b-b154-0daee52d1e62 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:21.136098 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189743d9-8329-46cd-9cee-b92f073b1a26 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:21.152627 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad64adc-e2d7-4aa0-8821-ecdbb0581b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:21.153185 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad64adc-e2d7-4aa0-8821-ecdbb0581b61 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:21.153467 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad64adc-e2d7-4aa0-8821-ecdbb0581b61 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:21.154298 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad64adc-e2d7-4aa0-8821-ecdbb0581b61 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:21.155276 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad64adc-e2d7-4aa0-8821-ecdbb0581b61 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:21.155647 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad64adc-e2d7-4aa0-8821-ecdbb0581b61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:21.157055 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad64adc-e2d7-4aa0-8821-ecdbb0581b61 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:21.170058 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 538/1093] 10.222.0.50 () {56 vars in 1115 bytes} [Fri May 27 15:02:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:02:21.171018 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad64adc-e2d7-4aa0-8821-ecdbb0581b61 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:21.172575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c66fe3-21b3-465f-be05-78baaf08de4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:21.173223 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c66fe3-21b3-465f-be05-78baaf08de4f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:21.173510 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c66fe3-21b3-465f-be05-78baaf08de4f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:21.173785 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c66fe3-21b3-465f-be05-78baaf08de4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:21.216131 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:21.240546 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c66fe3-21b3-465f-be05-78baaf08de4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:21.242007 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c66fe3-21b3-465f-be05-78baaf08de4f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:21.249130 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:21.271188 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad64adc-e2d7-4aa0-8821-ecdbb0581b61 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:21.303770 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d0754-3ff5-40cf-b929-f127ceffcf49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:21.304555 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d0754-3ff5-40cf-b929-f127ceffcf49 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:21.304846 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d0754-3ff5-40cf-b929-f127ceffcf49 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:21.305127 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612d0754-3ff5-40cf-b929-f127ceffcf49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:21.317009 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612d0754-3ff5-40cf-b929-f127ceffcf49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:21.318455 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612d0754-3ff5-40cf-b929-f127ceffcf49 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:21.335550 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-612d0754-3ff5-40cf-b929-f127ceffcf49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:02:21.335550 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:21.335550 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:02:21.335550 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:02:21.335550 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:02:21.335550 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:02:21.335550 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:02:21.335550 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:02:21.335550 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:02:21.335550 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:02:21.335550 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:02:21.335550 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:02:21.335550 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:02:21.335550 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:02:21.335550 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:02:21.336650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:02:21.336650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:02:21.336650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:02:21.336650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:02:21.336650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:02:21.336650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:02:21.336650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:02:21.336650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:02:21.336650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:02:21.336650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:02:21.336650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:02:21.336650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:02:21.336650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:02:21.336650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:02:21.336650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:02:21.336650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:02:21.336650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:02:21.337598 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:02:21.337598 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:02:21.337598 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:02:21.337598 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:02:21.337598 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 539/1094] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:02:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:02:21.340062 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 556/1095] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:02:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 27 15:02:21.342517 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d0754-3ff5-40cf-b929-f127ceffcf49 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:21.344610 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c66fe3-21b3-465f-be05-78baaf08de4f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:21.415214 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:21.440458 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612d0754-3ff5-40cf-b929-f127ceffcf49 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:21.440904 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c66fe3-21b3-465f-be05-78baaf08de4f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:21.468144 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:21.474019 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1aed1c-4cca-4f1e-b5d8-ca03842eed1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:21.474871 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf10722f-38e7-4e22-ac92-45ffc11dfd20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:21.475008 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1aed1c-4cca-4f1e-b5d8-ca03842eed1e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:21.475201 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1aed1c-4cca-4f1e-b5d8-ca03842eed1e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:21.475427 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1aed1c-4cca-4f1e-b5d8-ca03842eed1e None admin] PATH_INFO: `/v3/projects/31ce7d9017dc4804b6a367025d34990e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:21.475528 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10722f-38e7-4e22-ac92-45ffc11dfd20 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:21.475807 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10722f-38e7-4e22-ac92-45ffc11dfd20 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:21.476086 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf10722f-38e7-4e22-ac92-45ffc11dfd20 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:21.476597 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1aed1c-4cca-4f1e-b5d8-ca03842eed1e None admin] RBAC: Authorizing `identity:get_project(project_id=31ce7d9017dc4804b6a367025d34990e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:21.476794 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf10722f-38e7-4e22-ac92-45ffc11dfd20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:21.477104 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf10722f-38e7-4e22-ac92-45ffc11dfd20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:21.478003 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1aed1c-4cca-4f1e-b5d8-ca03842eed1e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:21.478448 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf10722f-38e7-4e22-ac92-45ffc11dfd20 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:21.479821 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 557/1096] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:02:21 2022] GET /identity/v3/projects/31ce7d9017dc4804b6a367025d34990e => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:02:21.485606 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1aed1c-4cca-4f1e-b5d8-ca03842eed1e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:21.491453 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 540/1097] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:02:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:21.495861 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf10722f-38e7-4e22-ac92-45ffc11dfd20 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:21.583042 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1aed1c-4cca-4f1e-b5d8-ca03842eed1e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:21.593973 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf10722f-38e7-4e22-ac92-45ffc11dfd20 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:21.616402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd5fe40-3217-4448-ba59-493e0ff89916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:21.617042 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd5fe40-3217-4448-ba59-493e0ff89916 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:21.617318 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd5fe40-3217-4448-ba59-493e0ff89916 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:21.617592 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd5fe40-3217-4448-ba59-493e0ff89916 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:21.618415 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd5fe40-3217-4448-ba59-493e0ff89916 None admin] RBAC: Adding query filter params (role.id=759d845eb6284e42931b4265af448ebc, scope.project.id=31ce7d9017dc4804b6a367025d34990e, user.id=613920d770f541b6b96901bd56b268db) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:21.618720 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd5fe40-3217-4448-ba59-493e0ff89916 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:21.620149 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd5fe40-3217-4448-ba59-493e0ff89916 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:21.627137 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 558/1098] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:02:21 2022] GET /identity/v3/role_assignments?user.id=613920d770f541b6b96901bd56b268db&scope.project.id=31ce7d9017dc4804b6a367025d34990e&role.id=759d845eb6284e42931b4265af448ebc => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:02:21.628048 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4362b0-0bdb-42e0-9aa8-0b15fa4f09ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:21.628908 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4362b0-0bdb-42e0-9aa8-0b15fa4f09ea None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:21.629191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4362b0-0bdb-42e0-9aa8-0b15fa4f09ea None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:21.629468 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4362b0-0bdb-42e0-9aa8-0b15fa4f09ea None admin] PATH_INFO: `/v3/users/nova` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:21.637209 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4362b0-0bdb-42e0-9aa8-0b15fa4f09ea None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:21.638924 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4362b0-0bdb-42e0-9aa8-0b15fa4f09ea None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:21.654786 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-2a4362b0-0bdb-42e0-9aa8-0b15fa4f09ea None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 27 15:02:21.654786 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:21.654786 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:02:21.654786 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:02:21.654786 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:02:21.654786 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:02:21.654786 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:02:21.654786 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:02:21.654786 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:02:21.654786 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:02:21.654786 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:02:21.654786 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:02:21.654786 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 15:02:21.654786 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 15:02:21.654786 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 15:02:21.654786 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 15:02:21.656089 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:02:21.656089 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:02:21.656089 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 15:02:21.656089 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:02:21.656089 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 15:02:21.656089 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:02:21.656089 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:02:21.656089 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:02:21.656089 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:02:21.656089 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:02:21.656089 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:02:21.656089 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:02:21.656089 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:02:21.656089 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:02:21.656089 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:02:21.656089 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:02:21.656089 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:02:21.656089 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:02:21.657309 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:02:21.657309 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:02:21.657309 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 15:02:21.657309 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 15:02:21.657309 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 15:02:21.657309 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 15:02:21.657309 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 15:02:21.657309 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 15:02:21.657309 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 27 15:02:21.657309 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:02:21.659046 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 541/1099] 10.222.0.50 () {56 vars in 1093 bytes} [Fri May 27 15:02:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:02:21.662221 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd5fe40-3217-4448-ba59-493e0ff89916 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:21.671970 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:21.768682 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd5fe40-3217-4448-ba59-493e0ff89916 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:21.803403 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971eeeb9-71fd-4215-8c81-a74acd61be60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:21.804077 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971eeeb9-71fd-4215-8c81-a74acd61be60 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:21.804357 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971eeeb9-71fd-4215-8c81-a74acd61be60 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:21.804630 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971eeeb9-71fd-4215-8c81-a74acd61be60 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:21.805475 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971eeeb9-71fd-4215-8c81-a74acd61be60 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:21.805783 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971eeeb9-71fd-4215-8c81-a74acd61be60 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:21.807237 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971eeeb9-71fd-4215-8c81-a74acd61be60 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:21.825607 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 559/1100] 10.222.0.50 () {56 vars in 1133 bytes} [Fri May 27 15:02:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:21.831451 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4362b0-0bdb-42e0-9aa8-0b15fa4f09ea None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:21.884192 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:21.932102 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4362b0-0bdb-42e0-9aa8-0b15fa4f09ea None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:21.970701 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eed6b2c-13ca-4974-a19a-140a6f7647e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:21.971380 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eed6b2c-13ca-4974-a19a-140a6f7647e9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:21.971657 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eed6b2c-13ca-4974-a19a-140a6f7647e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:21.971928 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eed6b2c-13ca-4974-a19a-140a6f7647e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:22.037758 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eed6b2c-13ca-4974-a19a-140a6f7647e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:22.039203 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eed6b2c-13ca-4974-a19a-140a6f7647e9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:22.085585 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:22.137506 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 542/1101] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:02:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 15:02:22.143417 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971eeeb9-71fd-4215-8c81-a74acd61be60 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:22.207131 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:22.245541 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971eeeb9-71fd-4215-8c81-a74acd61be60 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:22.278450 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b759342a-4726-4fea-99c3-054d996e3dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:22.279853 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b759342a-4726-4fea-99c3-054d996e3dc3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:22.280137 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b759342a-4726-4fea-99c3-054d996e3dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:22.280413 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b759342a-4726-4fea-99c3-054d996e3dc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:22.290225 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b759342a-4726-4fea-99c3-054d996e3dc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:22.291629 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b759342a-4726-4fea-99c3-054d996e3dc3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:22.309590 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-b759342a-4726-4fea-99c3-054d996e3dc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:02:22.309590 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:22.309590 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:02:22.309590 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:02:22.309590 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:02:22.309590 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:02:22.309590 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:02:22.309590 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:02:22.309590 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:02:22.309590 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:02:22.309590 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:02:22.309590 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:02:22.309590 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:02:22.309590 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:02:22.309590 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:02:22.310530 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:02:22.310530 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:02:22.310530 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:02:22.310530 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:02:22.310530 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:02:22.310530 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:02:22.310530 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:02:22.310530 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:02:22.310530 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:02:22.310530 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:02:22.310530 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:02:22.310530 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:02:22.310530 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:02:22.310530 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:02:22.310530 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:02:22.310530 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:02:22.310530 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:02:22.311940 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:02:22.311940 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:02:22.311940 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:02:22.311940 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:02:22.311940 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 560/1102] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:02:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:02:22.316390 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eed6b2c-13ca-4974-a19a-140a6f7647e9 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:22.416663 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eed6b2c-13ca-4974-a19a-140a6f7647e9 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:22.448990 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ded10-5bb2-43f5-a634-92a91386133c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:22.449647 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ded10-5bb2-43f5-a634-92a91386133c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:22.449932 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ded10-5bb2-43f5-a634-92a91386133c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:22.450212 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ded10-5bb2-43f5-a634-92a91386133c None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:22.451590 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1ded10-5bb2-43f5-a634-92a91386133c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:22.451900 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1ded10-5bb2-43f5-a634-92a91386133c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:22.453258 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1ded10-5bb2-43f5-a634-92a91386133c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:22.464600 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 543/1103] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:02:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:22.471224 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b759342a-4726-4fea-99c3-054d996e3dc3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:22.567075 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b759342a-4726-4fea-99c3-054d996e3dc3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:22.583364 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:22.601933 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2020ee-bdf1-4f32-b5f7-fb923fdf705e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:22.602713 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2020ee-bdf1-4f32-b5f7-fb923fdf705e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:22.603026 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2020ee-bdf1-4f32-b5f7-fb923fdf705e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:22.603306 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2020ee-bdf1-4f32-b5f7-fb923fdf705e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:22.609986 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2020ee-bdf1-4f32-b5f7-fb923fdf705e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:22.611444 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2020ee-bdf1-4f32-b5f7-fb923fdf705e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:22.625197 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-be2020ee-bdf1-4f32-b5f7-fb923fdf705e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:02:22.625197 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:22.625197 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:02:22.625197 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:02:22.625197 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:02:22.625197 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:02:22.625197 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:02:22.625197 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:02:22.625197 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:02:22.625197 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:02:22.625197 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:02:22.625197 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:02:22.625197 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 15:02:22.625197 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 15:02:22.625197 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 15:02:22.626256 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 15:02:22.626256 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:02:22.626256 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:02:22.626256 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:02:22.626256 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:02:22.626256 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:02:22.626256 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:02:22.626256 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:02:22.626256 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:02:22.626256 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:02:22.626256 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:02:22.626256 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:02:22.626256 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:02:22.626256 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:02:22.626256 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:02:22.626256 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:02:22.626256 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:02:22.627257 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 15:02:22.627257 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 15:02:22.627257 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 15:02:22.627257 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 15:02:22.627257 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 15:02:22.627257 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 15:02:22.627257 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:02:22.627257 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:02:22.627257 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 561/1104] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:02:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:02:22.632250 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ded10-5bb2-43f5-a634-92a91386133c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:22.645286 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:22.730062 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ded10-5bb2-43f5-a634-92a91386133c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:22.764633 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee568060-84d8-48b8-8db6-4195c581ba65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:22.765315 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee568060-84d8-48b8-8db6-4195c581ba65 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:22.765603 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee568060-84d8-48b8-8db6-4195c581ba65 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:22.765886 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee568060-84d8-48b8-8db6-4195c581ba65 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:22.766688 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee568060-84d8-48b8-8db6-4195c581ba65 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:22.767040 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee568060-84d8-48b8-8db6-4195c581ba65 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:22.768525 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee568060-84d8-48b8-8db6-4195c581ba65 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:22.781312 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 544/1105] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 15:02:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:02:22.787491 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2020ee-bdf1-4f32-b5f7-fb923fdf705e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:22.885128 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2020ee-bdf1-4f32-b5f7-fb923fdf705e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:22.917963 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adf1fb8-ec10-4984-8853-0be639f815ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:22.918610 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adf1fb8-ec10-4984-8853-0be639f815ec None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:22.918922 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adf1fb8-ec10-4984-8853-0be639f815ec None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:22.919322 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adf1fb8-ec10-4984-8853-0be639f815ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:22.920254 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adf1fb8-ec10-4984-8853-0be639f815ec None admin] RBAC: Adding query filter params (role.id=4564db14e2a543f18cb88a5bea5aada2, scope.project.id=daf12585428b41a2b45bdbc85413a6c4, user.id=451a9232f95e494683d81b6078a318d0) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:22.920568 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adf1fb8-ec10-4984-8853-0be639f815ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:22.921987 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adf1fb8-ec10-4984-8853-0be639f815ec None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:22.928094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 562/1106] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:02:22 2022] GET /identity/v3/role_assignments?user.id=451a9232f95e494683d81b6078a318d0&scope.project.id=daf12585428b41a2b45bdbc85413a6c4&role.id=4564db14e2a543f18cb88a5bea5aada2 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:23.876186 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05502ba1-b1f0-4583-90e1-4e826438ef88 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:23.876738 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05502ba1-b1f0-4583-90e1-4e826438ef88 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:23.876738 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05502ba1-b1f0-4583-90e1-4e826438ef88 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:23.878103 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 545/1107] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:02:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:02:23.883216 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877f45a-64ac-43b3-a6ee-3bc1bed40d3a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:23.883634 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877f45a-64ac-43b3-a6ee-3bc1bed40d3a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:23.883783 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8877f45a-64ac-43b3-a6ee-3bc1bed40d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:23.900834 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8877f45a-64ac-43b3-a6ee-3bc1bed40d3a None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:02:23.902006 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8877f45a-64ac-43b3-a6ee-3bc1bed40d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:02:23.935455 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:23.968415 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 563/1108] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:02:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:02:23.992034 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05502ba1-b1f0-4583-90e1-4e826438ef88 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:24.059564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:24.090098 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05502ba1-b1f0-4583-90e1-4e826438ef88 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:24.122563 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05821f95-5d2c-4865-ac50-aa542ba29de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:24.123349 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05821f95-5d2c-4865-ac50-aa542ba29de7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:24.123629 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05821f95-5d2c-4865-ac50-aa542ba29de7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:24.123902 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05821f95-5d2c-4865-ac50-aa542ba29de7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:24.125043 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05821f95-5d2c-4865-ac50-aa542ba29de7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:24.126505 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05821f95-5d2c-4865-ac50-aa542ba29de7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:24.128061 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 546/1109] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:02:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:24.133678 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8877f45a-64ac-43b3-a6ee-3bc1bed40d3a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:24.230998 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8877f45a-64ac-43b3-a6ee-3bc1bed40d3a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:24.264885 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2064de0-4fd1-4a7b-89da-db00a3374953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:24.265539 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2064de0-4fd1-4a7b-89da-db00a3374953 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:24.265822 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2064de0-4fd1-4a7b-89da-db00a3374953 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:24.266101 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2064de0-4fd1-4a7b-89da-db00a3374953 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:24.266808 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2064de0-4fd1-4a7b-89da-db00a3374953 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:24.268257 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2064de0-4fd1-4a7b-89da-db00a3374953 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:24.286839 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 564/1110] 10.222.0.50 () {60 vars in 1132 bytes} [Fri May 27 15:02:24 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:02:26.501574 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc3ef78-f328-4907-a24f-c5595b9c6bd6 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:26.502097 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc3ef78-f328-4907-a24f-c5595b9c6bd6 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:26.502097 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc3ef78-f328-4907-a24f-c5595b9c6bd6 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:26.503430 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 547/1111] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:02:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:02:26.508640 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4961a41-2c1c-4b1f-b8c6-705271f29a54 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:26.509019 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4961a41-2c1c-4b1f-b8c6-705271f29a54 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:26.509198 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4961a41-2c1c-4b1f-b8c6-705271f29a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:26.526142 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d4961a41-2c1c-4b1f-b8c6-705271f29a54 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:02:26.527357 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d4961a41-2c1c-4b1f-b8c6-705271f29a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:02:26.546527 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:26.593325 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 565/1112] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:02:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:02:26.616133 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc3ef78-f328-4907-a24f-c5595b9c6bd6 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:26.714326 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc3ef78-f328-4907-a24f-c5595b9c6bd6 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:26.746407 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e0cd3-f17c-4b0f-a388-49fa044210e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:26.747898 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e0cd3-f17c-4b0f-a388-49fa044210e0 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:26.748259 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e0cd3-f17c-4b0f-a388-49fa044210e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:26.748536 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e0cd3-f17c-4b0f-a388-49fa044210e0 None admin] PATH_INFO: `/v3/roles/ad9498a5e2064a468acc151fb2bce49e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:26.749975 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e0cd3-f17c-4b0f-a388-49fa044210e0 None admin] RBAC: Authorizing `identity:get_role(role_id=ad9498a5e2064a468acc151fb2bce49e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:26.751361 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e0cd3-f17c-4b0f-a388-49fa044210e0 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:26.752753 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 548/1113] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:02:26 2022] GET /identity/v3/roles/ad9498a5e2064a468acc151fb2bce49e => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:26.758302 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4961a41-2c1c-4b1f-b8c6-705271f29a54 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:26.856085 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4961a41-2c1c-4b1f-b8c6-705271f29a54 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:26.905265 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2949963a-08c8-4fc7-8caf-f78c60f1835e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:26.906129 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949963a-08c8-4fc7-8caf-f78c60f1835e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:26.906407 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949963a-08c8-4fc7-8caf-f78c60f1835e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:26.906679 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949963a-08c8-4fc7-8caf-f78c60f1835e None admin] PATH_INFO: `/v3/users/09b7038edc9d4117bb9db14706aa1664` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:26.920270 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2949963a-08c8-4fc7-8caf-f78c60f1835e None admin] RBAC: Authorizing `identity:get_user(user_id=09b7038edc9d4117bb9db14706aa1664)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:26.921719 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2949963a-08c8-4fc7-8caf-f78c60f1835e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:26.923401 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 566/1114] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:02:26 2022] GET /identity/v3/users/09b7038edc9d4117bb9db14706aa1664 => generated 286 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:26.929179 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e0cd3-f17c-4b0f-a388-49fa044210e0 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:26.938684 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:27.025926 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e0cd3-f17c-4b0f-a388-49fa044210e0 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:27.058286 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77acd4e4-aab0-4f2e-8f1c-3224bea5e9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:27.058947 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77acd4e4-aab0-4f2e-8f1c-3224bea5e9c9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:27.059267 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77acd4e4-aab0-4f2e-8f1c-3224bea5e9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:27.059592 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77acd4e4-aab0-4f2e-8f1c-3224bea5e9c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:27.125326 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77acd4e4-aab0-4f2e-8f1c-3224bea5e9c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:27.126770 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77acd4e4-aab0-4f2e-8f1c-3224bea5e9c9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:27.134131 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:27.226872 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 549/1115] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:02:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 15:02:27.234305 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2949963a-08c8-4fc7-8caf-f78c60f1835e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:27.251541 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:27.333121 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2949963a-08c8-4fc7-8caf-f78c60f1835e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:27.371144 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca41e082-3317-4631-9708-0b90ff4cf2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:27.371993 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41e082-3317-4631-9708-0b90ff4cf2a7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:27.372302 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41e082-3317-4631-9708-0b90ff4cf2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:27.372576 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41e082-3317-4631-9708-0b90ff4cf2a7 None admin] PATH_INFO: `/v3/projects/31ce7d9017dc4804b6a367025d34990e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:27.373762 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca41e082-3317-4631-9708-0b90ff4cf2a7 None admin] RBAC: Authorizing `identity:get_project(project_id=31ce7d9017dc4804b6a367025d34990e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:27.375213 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca41e082-3317-4631-9708-0b90ff4cf2a7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:27.376912 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 567/1116] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:02:27 2022] GET /identity/v3/projects/31ce7d9017dc4804b6a367025d34990e => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:27.382979 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77acd4e4-aab0-4f2e-8f1c-3224bea5e9c9 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:27.479616 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77acd4e4-aab0-4f2e-8f1c-3224bea5e9c9 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:27.487483 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:27.513196 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7be2ea7-7cd4-4641-b048-28e4cb336d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:27.513838 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7be2ea7-7cd4-4641-b048-28e4cb336d79 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:27.514125 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7be2ea7-7cd4-4641-b048-28e4cb336d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:27.514407 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7be2ea7-7cd4-4641-b048-28e4cb336d79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:27.515265 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7be2ea7-7cd4-4641-b048-28e4cb336d79 None admin] RBAC: Adding query filter params (role.id=ad9498a5e2064a468acc151fb2bce49e, scope.project.id=31ce7d9017dc4804b6a367025d34990e, user.id=09b7038edc9d4117bb9db14706aa1664) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:27.515583 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7be2ea7-7cd4-4641-b048-28e4cb336d79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:27.517003 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7be2ea7-7cd4-4641-b048-28e4cb336d79 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:27.522302 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 550/1117] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:02:27 2022] GET /identity/v3/role_assignments?user.id=09b7038edc9d4117bb9db14706aa1664&scope.project.id=31ce7d9017dc4804b6a367025d34990e&role.id=ad9498a5e2064a468acc151fb2bce49e => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:29.707267 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d878df52-86ca-4322-b3da-77cbb0f01f6a None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:29.707819 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d878df52-86ca-4322-b3da-77cbb0f01f6a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:29.707819 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d878df52-86ca-4322-b3da-77cbb0f01f6a None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:29.709195 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 568/1118] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:02:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:02:29.714733 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c1b14-3f07-4ebb-8820-e37495e8b0e8 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:29.715147 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c1b14-3f07-4ebb-8820-e37495e8b0e8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:29.715324 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2c1b14-3f07-4ebb-8820-e37495e8b0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:29.731690 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ee2c1b14-3f07-4ebb-8820-e37495e8b0e8 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:02:29.732862 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ee2c1b14-3f07-4ebb-8820-e37495e8b0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:02:29.797180 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 551/1119] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:02:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:02:29.819545 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d878df52-86ca-4322-b3da-77cbb0f01f6a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:29.899193 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:29.917745 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d878df52-86ca-4322-b3da-77cbb0f01f6a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:29.951943 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81441486-faaf-4228-831a-d184d8f7c865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:29.952787 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81441486-faaf-4228-831a-d184d8f7c865 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:29.953073 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81441486-faaf-4228-831a-d184d8f7c865 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:29.953350 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81441486-faaf-4228-831a-d184d8f7c865 None admin] PATH_INFO: `/v3/roles/ad9498a5e2064a468acc151fb2bce49e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:29.954807 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81441486-faaf-4228-831a-d184d8f7c865 None admin] RBAC: Authorizing `identity:get_role(role_id=ad9498a5e2064a468acc151fb2bce49e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:29.956190 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81441486-faaf-4228-831a-d184d8f7c865 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:29.957615 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 569/1120] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:02:29 2022] GET /identity/v3/roles/ad9498a5e2064a468acc151fb2bce49e => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:29.963301 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c1b14-3f07-4ebb-8820-e37495e8b0e8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:30.055177 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:30.057998 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2c1b14-3f07-4ebb-8820-e37495e8b0e8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:30.092496 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee5cdfe-9ae7-4818-9657-801ad49e41be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:30.093338 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5cdfe-9ae7-4818-9657-801ad49e41be None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:30.093622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5cdfe-9ae7-4818-9657-801ad49e41be None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:30.093898 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5cdfe-9ae7-4818-9657-801ad49e41be None admin] PATH_INFO: `/v3/users/09b7038edc9d4117bb9db14706aa1664` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:30.095088 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee5cdfe-9ae7-4818-9657-801ad49e41be None admin] RBAC: Authorizing `identity:get_user(user_id=09b7038edc9d4117bb9db14706aa1664)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:30.096517 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee5cdfe-9ae7-4818-9657-801ad49e41be None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:30.098122 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 552/1121] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:02:29 2022] GET /identity/v3/users/09b7038edc9d4117bb9db14706aa1664 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:30.103827 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81441486-faaf-4228-831a-d184d8f7c865 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:30.199277 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81441486-faaf-4228-831a-d184d8f7c865 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:30.232873 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf1a798-6ed9-4d1f-932f-9f6d5f7ad9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:30.233658 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1a798-6ed9-4d1f-932f-9f6d5f7ad9e2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:30.233940 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1a798-6ed9-4d1f-932f-9f6d5f7ad9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:30.234214 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1a798-6ed9-4d1f-932f-9f6d5f7ad9e2 None admin] PATH_INFO: `/v3/projects/31ce7d9017dc4804b6a367025d34990e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:30.235403 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf1a798-6ed9-4d1f-932f-9f6d5f7ad9e2 None admin] RBAC: Authorizing `identity:get_project(project_id=31ce7d9017dc4804b6a367025d34990e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:30.236817 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf1a798-6ed9-4d1f-932f-9f6d5f7ad9e2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:30.238492 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 570/1122] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:02:30 2022] GET /identity/v3/projects/31ce7d9017dc4804b6a367025d34990e => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:30.244320 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee5cdfe-9ae7-4818-9657-801ad49e41be None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:30.339020 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee5cdfe-9ae7-4818-9657-801ad49e41be None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:30.371661 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674fcf57-afa5-47d7-ae8c-e4931dd35334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:30.372683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674fcf57-afa5-47d7-ae8c-e4931dd35334 None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:30.372964 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674fcf57-afa5-47d7-ae8c-e4931dd35334 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:30.373242 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674fcf57-afa5-47d7-ae8c-e4931dd35334 None admin] PATH_INFO: `/v3/projects/31ce7d9017dc4804b6a367025d34990e/users/09b7038edc9d4117bb9db14706aa1664/roles/ad9498a5e2064a468acc151fb2bce49e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:30.375424 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674fcf57-afa5-47d7-ae8c-e4931dd35334 None admin] RBAC: Authorizing `identity:create_grant(project_id=31ce7d9017dc4804b6a367025d34990e, user_id=09b7038edc9d4117bb9db14706aa1664, role_id=ad9498a5e2064a468acc151fb2bce49e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:30.376850 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674fcf57-afa5-47d7-ae8c-e4931dd35334 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:30.389254 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 553/1123] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 15:02:30 2022] PUT /identity/v3/projects/31ce7d9017dc4804b6a367025d34990e/users/09b7038edc9d4117bb9db14706aa1664/roles/ad9498a5e2064a468acc151fb2bce49e => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:02:32.539925 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa45e3d-df9e-4ad5-81c1-f5753ceaeec5 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:32.540457 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa45e3d-df9e-4ad5-81c1-f5753ceaeec5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:32.540457 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa45e3d-df9e-4ad5-81c1-f5753ceaeec5 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:32.541847 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 571/1124] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:02:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:02:32.549760 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e9e2f-a916-40f8-bdee-fb2ddef495e5 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:32.550156 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e9e2f-a916-40f8-bdee-fb2ddef495e5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:32.550320 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e9e2f-a916-40f8-bdee-fb2ddef495e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:32.566723 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-145e9e2f-a916-40f8-bdee-fb2ddef495e5 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:02:32.567926 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-145e9e2f-a916-40f8-bdee-fb2ddef495e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:02:32.589634 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:32.632695 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 554/1125] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:02:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:02:32.656335 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa45e3d-df9e-4ad5-81c1-f5753ceaeec5 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:32.675244 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:32.755107 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa45e3d-df9e-4ad5-81c1-f5753ceaeec5 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:32.787393 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991aca91-0f33-4012-826a-f32608e74189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:32.788231 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991aca91-0f33-4012-826a-f32608e74189 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:32.788517 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991aca91-0f33-4012-826a-f32608e74189 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:32.788793 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991aca91-0f33-4012-826a-f32608e74189 None admin] PATH_INFO: `/v3/roles/ad9498a5e2064a468acc151fb2bce49e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:32.790294 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991aca91-0f33-4012-826a-f32608e74189 None admin] RBAC: Authorizing `identity:get_role(role_id=ad9498a5e2064a468acc151fb2bce49e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:32.791636 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991aca91-0f33-4012-826a-f32608e74189 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:32.793001 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 572/1126] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:02:32 2022] GET /identity/v3/roles/ad9498a5e2064a468acc151fb2bce49e => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:32.799306 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145e9e2f-a916-40f8-bdee-fb2ddef495e5 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:32.895676 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145e9e2f-a916-40f8-bdee-fb2ddef495e5 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:32.928741 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb95968-c796-461d-8e46-e6bedff8a54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:32.929587 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb95968-c796-461d-8e46-e6bedff8a54e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:32.929870 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb95968-c796-461d-8e46-e6bedff8a54e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:32.930144 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb95968-c796-461d-8e46-e6bedff8a54e None admin] PATH_INFO: `/v3/users/09b7038edc9d4117bb9db14706aa1664` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:32.931338 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb95968-c796-461d-8e46-e6bedff8a54e None admin] RBAC: Authorizing `identity:get_user(user_id=09b7038edc9d4117bb9db14706aa1664)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:32.932772 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb95968-c796-461d-8e46-e6bedff8a54e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:32.934516 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 555/1127] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:02:32 2022] GET /identity/v3/users/09b7038edc9d4117bb9db14706aa1664 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:32.939946 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991aca91-0f33-4012-826a-f32608e74189 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:33.024700 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:33.039765 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991aca91-0f33-4012-826a-f32608e74189 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:33.071835 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bea71cc-74fc-4332-85c5-dabfaece143d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:33.072400 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea71cc-74fc-4332-85c5-dabfaece143d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:33.072678 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea71cc-74fc-4332-85c5-dabfaece143d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:33.072945 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea71cc-74fc-4332-85c5-dabfaece143d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:33.136532 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bea71cc-74fc-4332-85c5-dabfaece143d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:33.138473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bea71cc-74fc-4332-85c5-dabfaece143d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:33.218679 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:33.234108 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 573/1128] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:02:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 15:02:33.239904 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb95968-c796-461d-8e46-e6bedff8a54e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:33.267792 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:33.337593 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb95968-c796-461d-8e46-e6bedff8a54e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:33.369869 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266e0c69-cdae-458d-a624-cb4bf1ced64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:33.370712 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e0c69-cdae-458d-a624-cb4bf1ced64d None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:33.371026 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e0c69-cdae-458d-a624-cb4bf1ced64d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:33.371305 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266e0c69-cdae-458d-a624-cb4bf1ced64d None admin] PATH_INFO: `/v3/projects/31ce7d9017dc4804b6a367025d34990e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:33.373191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266e0c69-cdae-458d-a624-cb4bf1ced64d None admin] RBAC: Authorizing `identity:get_project(project_id=31ce7d9017dc4804b6a367025d34990e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:33.374626 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266e0c69-cdae-458d-a624-cb4bf1ced64d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:33.376338 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 556/1129] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:02:33 2022] GET /identity/v3/projects/31ce7d9017dc4804b6a367025d34990e => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:33.382415 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bea71cc-74fc-4332-85c5-dabfaece143d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:33.477854 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bea71cc-74fc-4332-85c5-dabfaece143d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:33.510802 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf2d93b-9bd0-484a-8eba-0d06c5d2f26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:33.511474 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2d93b-9bd0-484a-8eba-0d06c5d2f26a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:33.511762 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2d93b-9bd0-484a-8eba-0d06c5d2f26a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:33.512037 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2d93b-9bd0-484a-8eba-0d06c5d2f26a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:33.513479 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf2d93b-9bd0-484a-8eba-0d06c5d2f26a None admin] RBAC: Adding query filter params (role.id=ad9498a5e2064a468acc151fb2bce49e, scope.project.id=31ce7d9017dc4804b6a367025d34990e, user.id=09b7038edc9d4117bb9db14706aa1664) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:33.513787 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf2d93b-9bd0-484a-8eba-0d06c5d2f26a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:33.515225 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf2d93b-9bd0-484a-8eba-0d06c5d2f26a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:33.520886 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 574/1130] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:02:33 2022] GET /identity/v3/role_assignments?user.id=09b7038edc9d4117bb9db14706aa1664&scope.project.id=31ce7d9017dc4804b6a367025d34990e&role.id=ad9498a5e2064a468acc151fb2bce49e => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:35.738573 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648a61ec-cb48-439d-adba-962fde58de9c None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:35.739189 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648a61ec-cb48-439d-adba-962fde58de9c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:35.739189 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648a61ec-cb48-439d-adba-962fde58de9c None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:35.740542 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 557/1131] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:02:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:02:35.745777 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed2bbc-7644-444c-99f5-b1fc030e70c3 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:35.746167 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed2bbc-7644-444c-99f5-b1fc030e70c3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:35.746338 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ed2bbc-7644-444c-99f5-b1fc030e70c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:35.763995 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d0ed2bbc-7644-444c-99f5-b1fc030e70c3 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:02:35.765160 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d0ed2bbc-7644-444c-99f5-b1fc030e70c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:02:35.809088 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:35.829457 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 575/1132] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:02:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:02:35.852960 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648a61ec-cb48-439d-adba-962fde58de9c None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:35.949733 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:35.971792 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648a61ec-cb48-439d-adba-962fde58de9c None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:36.005339 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a031dfb2-6305-4e04-96d6-69ea7c773295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:36.006046 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031dfb2-6305-4e04-96d6-69ea7c773295 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:36.006324 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031dfb2-6305-4e04-96d6-69ea7c773295 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:36.006596 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031dfb2-6305-4e04-96d6-69ea7c773295 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:36.007767 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a031dfb2-6305-4e04-96d6-69ea7c773295 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:36.009181 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a031dfb2-6305-4e04-96d6-69ea7c773295 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:36.010773 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 558/1133] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:02:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:36.016838 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ed2bbc-7644-444c-99f5-b1fc030e70c3 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:36.112941 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ed2bbc-7644-444c-99f5-b1fc030e70c3 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:36.146595 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37883044-4ec4-44bb-9525-03e1db00966c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:36.147285 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37883044-4ec4-44bb-9525-03e1db00966c None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:36.147572 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37883044-4ec4-44bb-9525-03e1db00966c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:36.147851 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37883044-4ec4-44bb-9525-03e1db00966c None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:36.148556 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37883044-4ec4-44bb-9525-03e1db00966c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:36.150004 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37883044-4ec4-44bb-9525-03e1db00966c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:36.175415 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 576/1134] 10.222.0.50 () {60 vars in 1141 bytes} [Fri May 27 15:02:36 2022] POST /identity/v3/projects => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:02:38.380990 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1795b0b-5654-4d5e-a5eb-4fef17c0d892 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:38.381593 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1795b0b-5654-4d5e-a5eb-4fef17c0d892 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:38.381593 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1795b0b-5654-4d5e-a5eb-4fef17c0d892 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:38.382962 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 559/1135] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:02:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:02:38.388757 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaaf265-7ea7-4c16-98fd-d15eb816255b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:38.389172 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaaf265-7ea7-4c16-98fd-d15eb816255b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:38.389322 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaaf265-7ea7-4c16-98fd-d15eb816255b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:38.406431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-fdaaf265-7ea7-4c16-98fd-d15eb816255b None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:02:38.407585 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fdaaf265-7ea7-4c16-98fd-d15eb816255b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:02:38.432469 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:38.477699 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 577/1136] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:02:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 15:02:38.502831 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1795b0b-5654-4d5e-a5eb-4fef17c0d892 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:38.600826 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1795b0b-5654-4d5e-a5eb-4fef17c0d892 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:38.635829 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd2fb9f-a3b2-4194-a7c0-8d9d41f302e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:38.636604 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd2fb9f-a3b2-4194-a7c0-8d9d41f302e5 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:38.636886 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd2fb9f-a3b2-4194-a7c0-8d9d41f302e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:38.637157 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd2fb9f-a3b2-4194-a7c0-8d9d41f302e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:38.639253 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd2fb9f-a3b2-4194-a7c0-8d9d41f302e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:38.640668 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd2fb9f-a3b2-4194-a7c0-8d9d41f302e5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:38.642262 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 560/1137] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:02:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:38.648121 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaaf265-7ea7-4c16-98fd-d15eb816255b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:38.742207 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaaf265-7ea7-4c16-98fd-d15eb816255b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:38.775348 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12e0840-c5c9-4956-b7de-2676e1cec6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:38.776007 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e0840-c5c9-4956-b7de-2676e1cec6b8 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:38.776291 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e0840-c5c9-4956-b7de-2676e1cec6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:38.776571 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12e0840-c5c9-4956-b7de-2676e1cec6b8 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:38.777274 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12e0840-c5c9-4956-b7de-2676e1cec6b8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:38.778955 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12e0840-c5c9-4956-b7de-2676e1cec6b8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:38.798749 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 578/1138] 10.222.0.50 () {60 vars in 1132 bytes} [Fri May 27 15:02:38 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:02:41.027239 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996f31f-faa2-4448-92f0-2815bda0d866 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:41.027804 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996f31f-faa2-4448-92f0-2815bda0d866 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:41.027804 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9996f31f-faa2-4448-92f0-2815bda0d866 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:41.029755 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 561/1139] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:02:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:02:41.035005 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de592edb-b0c5-49d6-9269-10cc46e84596 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:41.035402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de592edb-b0c5-49d6-9269-10cc46e84596 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:41.035563 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de592edb-b0c5-49d6-9269-10cc46e84596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:41.052473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-de592edb-b0c5-49d6-9269-10cc46e84596 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:02:41.053629 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-de592edb-b0c5-49d6-9269-10cc46e84596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:02:41.066193 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:41.119015 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 579/1140] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:02:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:02:41.142690 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9996f31f-faa2-4448-92f0-2815bda0d866 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:41.173007 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:41.245094 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9996f31f-faa2-4448-92f0-2815bda0d866 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:41.277075 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8b2710-4714-47cf-b389-c2bfa537df17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:41.277922 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b2710-4714-47cf-b389-c2bfa537df17 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:41.278752 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b2710-4714-47cf-b389-c2bfa537df17 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:41.279207 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b2710-4714-47cf-b389-c2bfa537df17 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:41.288778 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8b2710-4714-47cf-b389-c2bfa537df17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:02:41.296355 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-2a8b2710-4714-47cf-b389-c2bfa537df17 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:02:41.296355 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:41.296355 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:02:41.296355 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:02:41.296355 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:02:41.296355 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:02:41.296355 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:02:41.296355 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:02:41.296355 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:02:41.296355 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:02:41.296355 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:02:41.296355 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:02:41.296355 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:02:41.296355 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:02:41.296355 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:02:41.296355 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:02:41.297543 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:02:41.297543 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:02:41.297543 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:02:41.297543 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:02:41.297543 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:02:41.297543 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:02:41.297543 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:02:41.297543 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:02:41.297543 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:02:41.297543 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:02:41.297543 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:02:41.297543 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:02:41.297543 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:02:41.297543 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:02:41.297543 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:02:41.297543 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:02:41.298725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:02:41.298725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:02:41.298725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:02:41.298725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:02:41.298725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:02:41.298725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:02:41.298725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:02:41.298725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:02:41.298725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:02:41.298725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:02:41.298725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:02:41.298725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:02:41.298725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 562/1141] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 15:02:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:02:41.303540 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de592edb-b0c5-49d6-9269-10cc46e84596 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:41.397530 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de592edb-b0c5-49d6-9269-10cc46e84596 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:41.429736 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07db0e4a-707c-4a6b-80d9-6db1f5a721d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:41.430384 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db0e4a-707c-4a6b-80d9-6db1f5a721d9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:41.430665 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db0e4a-707c-4a6b-80d9-6db1f5a721d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:41.431043 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db0e4a-707c-4a6b-80d9-6db1f5a721d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:41.431784 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07db0e4a-707c-4a6b-80d9-6db1f5a721d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:41.432093 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07db0e4a-707c-4a6b-80d9-6db1f5a721d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:41.433445 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07db0e4a-707c-4a6b-80d9-6db1f5a721d9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:41.441799 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 580/1142] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:02:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:41.447338 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8b2710-4714-47cf-b389-c2bfa537df17 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:41.534036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:41.543943 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8b2710-4714-47cf-b389-c2bfa537df17 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:41.576660 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0cb93e-b888-48ca-84f8-fb13a64bec01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:41.577501 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0cb93e-b888-48ca-84f8-fb13a64bec01 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:41.577781 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0cb93e-b888-48ca-84f8-fb13a64bec01 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:41.578053 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0cb93e-b888-48ca-84f8-fb13a64bec01 None admin] PATH_INFO: `/v3/users/1dbe094c671c439e8e87024b53e5b125` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:41.591106 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0cb93e-b888-48ca-84f8-fb13a64bec01 None admin] RBAC: Authorizing `identity:get_user(user_id=1dbe094c671c439e8e87024b53e5b125)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:41.592532 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0cb93e-b888-48ca-84f8-fb13a64bec01 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:41.594139 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 563/1143] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:02:41 2022] GET /identity/v3/users/1dbe094c671c439e8e87024b53e5b125 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:41.599828 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07db0e4a-707c-4a6b-80d9-6db1f5a721d9 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:41.612380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:41.697916 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07db0e4a-707c-4a6b-80d9-6db1f5a721d9 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:41.730894 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c21ae4-c5e3-4bf6-8dee-89d2f8da5ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:41.731557 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c21ae4-c5e3-4bf6-8dee-89d2f8da5ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:41.731844 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c21ae4-c5e3-4bf6-8dee-89d2f8da5ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:41.732124 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c21ae4-c5e3-4bf6-8dee-89d2f8da5ae6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:41.797290 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c21ae4-c5e3-4bf6-8dee-89d2f8da5ae6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:41.798671 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c21ae4-c5e3-4bf6-8dee-89d2f8da5ae6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:41.810634 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:41.895138 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 581/1144] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:02:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 15:02:41.901165 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0cb93e-b888-48ca-84f8-fb13a64bec01 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:41.998757 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0cb93e-b888-48ca-84f8-fb13a64bec01 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:42.030340 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc24563-105f-4ccf-898d-8ff6f38e8fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:42.031207 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc24563-105f-4ccf-898d-8ff6f38e8fcc None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:42.031488 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc24563-105f-4ccf-898d-8ff6f38e8fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:42.031759 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc24563-105f-4ccf-898d-8ff6f38e8fcc None admin] PATH_INFO: `/v3/projects/b82f749e9dec4deeb6a308655841e0ee` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:42.032906 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc24563-105f-4ccf-898d-8ff6f38e8fcc None admin] RBAC: Authorizing `identity:get_project(project_id=b82f749e9dec4deeb6a308655841e0ee)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:42.034315 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc24563-105f-4ccf-898d-8ff6f38e8fcc None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:42.036009 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 564/1145] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:02:41 2022] GET /identity/v3/projects/b82f749e9dec4deeb6a308655841e0ee => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:02:42.042006 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c21ae4-c5e3-4bf6-8dee-89d2f8da5ae6 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:42.138936 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c21ae4-c5e3-4bf6-8dee-89d2f8da5ae6 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:42.149068 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:42.173108 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e3712b-2e44-4dc1-ad36-62d358cb1358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:42.173750 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3712b-2e44-4dc1-ad36-62d358cb1358 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:42.174032 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3712b-2e44-4dc1-ad36-62d358cb1358 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:42.174306 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3712b-2e44-4dc1-ad36-62d358cb1358 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:42.175158 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e3712b-2e44-4dc1-ad36-62d358cb1358 None admin] RBAC: Adding query filter params (role.id=759d845eb6284e42931b4265af448ebc, scope.project.id=b82f749e9dec4deeb6a308655841e0ee, user.id=1dbe094c671c439e8e87024b53e5b125) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:42.175468 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e3712b-2e44-4dc1-ad36-62d358cb1358 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:42.176878 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e3712b-2e44-4dc1-ad36-62d358cb1358 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:42.182206 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 582/1146] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:02:42 2022] GET /identity/v3/role_assignments?user.id=1dbe094c671c439e8e87024b53e5b125&scope.project.id=b82f749e9dec4deeb6a308655841e0ee&role.id=759d845eb6284e42931b4265af448ebc => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:44.379460 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e4d47-9ed0-4457-bfee-5de7d54edba1 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:44.379997 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e4d47-9ed0-4457-bfee-5de7d54edba1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:44.379997 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e4d47-9ed0-4457-bfee-5de7d54edba1 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:44.381372 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 565/1147] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:02:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:02:44.386636 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88b7320-0e4d-47e7-a894-930666316a0f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:44.387040 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88b7320-0e4d-47e7-a894-930666316a0f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:44.387754 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88b7320-0e4d-47e7-a894-930666316a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:44.404675 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e88b7320-0e4d-47e7-a894-930666316a0f None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:02:44.405794 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e88b7320-0e4d-47e7-a894-930666316a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:02:44.470499 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 583/1148] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:02:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:02:44.492422 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e4d47-9ed0-4457-bfee-5de7d54edba1 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:44.511941 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:44.593377 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e4d47-9ed0-4457-bfee-5de7d54edba1 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:44.625607 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f576084e-3cab-4b9c-9631-ba9d5326ac14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:44.626374 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576084e-3cab-4b9c-9631-ba9d5326ac14 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:44.626650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576084e-3cab-4b9c-9631-ba9d5326ac14 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:44.626951 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576084e-3cab-4b9c-9631-ba9d5326ac14 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:44.636535 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f576084e-3cab-4b9c-9631-ba9d5326ac14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:02:44.644691 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-f576084e-3cab-4b9c-9631-ba9d5326ac14 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:02:44.644691 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:44.644691 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:02:44.644691 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:02:44.644691 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:02:44.644691 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:02:44.644691 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:02:44.644691 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:02:44.644691 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:02:44.644691 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:02:44.644691 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:02:44.644691 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:02:44.644691 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:02:44.644691 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:02:44.644691 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:02:44.644691 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:02:44.645783 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:02:44.645783 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:02:44.645783 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:02:44.645783 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:02:44.645783 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:02:44.645783 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:02:44.645783 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:02:44.645783 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:02:44.645783 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:02:44.645783 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:02:44.645783 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:02:44.645783 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:02:44.645783 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:02:44.645783 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:02:44.645783 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:02:44.645783 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:02:44.646687 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:02:44.646687 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:02:44.646687 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:02:44.646687 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:02:44.646687 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:02:44.646687 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:02:44.646687 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:02:44.646687 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:02:44.646687 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:02:44.646687 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:02:44.646687 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:02:44.646687 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:02:44.646687 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 566/1149] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 15:02:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:02:44.652013 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88b7320-0e4d-47e7-a894-930666316a0f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:44.747388 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88b7320-0e4d-47e7-a894-930666316a0f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:44.750290 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:44.782028 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3168ce28-016e-4ceb-a5ce-b92513719e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:44.782689 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168ce28-016e-4ceb-a5ce-b92513719e9b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:44.783003 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168ce28-016e-4ceb-a5ce-b92513719e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:44.783280 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168ce28-016e-4ceb-a5ce-b92513719e9b None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:44.784025 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3168ce28-016e-4ceb-a5ce-b92513719e9b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:44.784331 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3168ce28-016e-4ceb-a5ce-b92513719e9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:44.785681 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3168ce28-016e-4ceb-a5ce-b92513719e9b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:44.793952 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 584/1150] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:02:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:44.799882 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f576084e-3cab-4b9c-9631-ba9d5326ac14 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:44.874693 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:44.896514 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f576084e-3cab-4b9c-9631-ba9d5326ac14 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:44.928445 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa317054-d3a6-4cc5-b9d4-e5589f12cddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:44.929304 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa317054-d3a6-4cc5-b9d4-e5589f12cddb None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:44.929593 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa317054-d3a6-4cc5-b9d4-e5589f12cddb None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:44.929871 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa317054-d3a6-4cc5-b9d4-e5589f12cddb None admin] PATH_INFO: `/v3/users/1dbe094c671c439e8e87024b53e5b125` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:44.931054 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa317054-d3a6-4cc5-b9d4-e5589f12cddb None admin] RBAC: Authorizing `identity:get_user(user_id=1dbe094c671c439e8e87024b53e5b125)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:44.932477 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa317054-d3a6-4cc5-b9d4-e5589f12cddb None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:44.934624 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 567/1151] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:02:44 2022] GET /identity/v3/users/1dbe094c671c439e8e87024b53e5b125 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:44.940398 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3168ce28-016e-4ceb-a5ce-b92513719e9b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:45.037712 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3168ce28-016e-4ceb-a5ce-b92513719e9b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:45.070578 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43b3ffd-8d2d-4283-b23b-a8a3189a3a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:45.071455 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b3ffd-8d2d-4283-b23b-a8a3189a3a8d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:45.071745 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b3ffd-8d2d-4283-b23b-a8a3189a3a8d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:45.072018 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43b3ffd-8d2d-4283-b23b-a8a3189a3a8d None admin] PATH_INFO: `/v3/projects/b82f749e9dec4deeb6a308655841e0ee` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:45.073185 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43b3ffd-8d2d-4283-b23b-a8a3189a3a8d None admin] RBAC: Authorizing `identity:get_project(project_id=b82f749e9dec4deeb6a308655841e0ee)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:45.074614 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43b3ffd-8d2d-4283-b23b-a8a3189a3a8d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:45.076370 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 585/1152] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:02:44 2022] GET /identity/v3/projects/b82f749e9dec4deeb6a308655841e0ee => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:45.082325 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa317054-d3a6-4cc5-b9d4-e5589f12cddb None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:45.180350 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa317054-d3a6-4cc5-b9d4-e5589f12cddb None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:45.212689 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4eafb8-d663-4d88-acb6-98f34d9a299d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:45.213816 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4eafb8-d663-4d88-acb6-98f34d9a299d None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:45.214093 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4eafb8-d663-4d88-acb6-98f34d9a299d None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:45.214361 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4eafb8-d663-4d88-acb6-98f34d9a299d None admin] PATH_INFO: `/v3/projects/b82f749e9dec4deeb6a308655841e0ee/users/1dbe094c671c439e8e87024b53e5b125/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:45.216970 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4eafb8-d663-4d88-acb6-98f34d9a299d None admin] RBAC: Authorizing `identity:create_grant(project_id=b82f749e9dec4deeb6a308655841e0ee, user_id=1dbe094c671c439e8e87024b53e5b125, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:45.218378 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4eafb8-d663-4d88-acb6-98f34d9a299d None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:45.230590 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 568/1153] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 15:02:45 2022] PUT /identity/v3/projects/b82f749e9dec4deeb6a308655841e0ee/users/1dbe094c671c439e8e87024b53e5b125/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:02:47.407961 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e9a731-6c5a-458e-85e4-b44064566549 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:47.408489 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e9a731-6c5a-458e-85e4-b44064566549 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:47.408489 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e9a731-6c5a-458e-85e4-b44064566549 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:47.409787 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 586/1154] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:02:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:02:47.416957 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e42fcb-99b5-47fa-9ff1-cde1d01f2656 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:47.416957 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e42fcb-99b5-47fa-9ff1-cde1d01f2656 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:47.416957 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e42fcb-99b5-47fa-9ff1-cde1d01f2656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:47.422574 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:47.437088 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-56e42fcb-99b5-47fa-9ff1-cde1d01f2656 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:02:47.438269 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-56e42fcb-99b5-47fa-9ff1-cde1d01f2656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:02:47.514753 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 569/1155] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:02:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:02:47.537712 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e9a731-6c5a-458e-85e4-b44064566549 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:47.577047 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:47.653126 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e9a731-6c5a-458e-85e4-b44064566549 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:47.690740 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9c6776-d6c4-4827-8a65-5d607dca3697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:47.691584 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c6776-d6c4-4827-8a65-5d607dca3697 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:47.691860 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c6776-d6c4-4827-8a65-5d607dca3697 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:47.692140 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9c6776-d6c4-4827-8a65-5d607dca3697 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:47.704555 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9c6776-d6c4-4827-8a65-5d607dca3697 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:02:47.712893 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-ce9c6776-d6c4-4827-8a65-5d607dca3697 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:02:47.712893 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:47.712893 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:02:47.712893 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:02:47.712893 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:02:47.712893 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:02:47.712893 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:02:47.712893 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:02:47.712893 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:02:47.712893 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:02:47.712893 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:02:47.712893 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:02:47.712893 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:02:47.712893 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:02:47.712893 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:02:47.712893 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:02:47.714270 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:02:47.714270 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:02:47.714270 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:02:47.714270 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:02:47.714270 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:02:47.714270 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:02:47.714270 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:02:47.714270 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:02:47.714270 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:02:47.714270 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:02:47.714270 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:02:47.714270 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:02:47.714270 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:02:47.714270 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:02:47.714270 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:02:47.714270 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:02:47.715658 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:02:47.715658 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:02:47.715658 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:02:47.715658 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:02:47.715658 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:02:47.715658 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:02:47.715658 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:02:47.715658 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:02:47.715658 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:02:47.715658 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:02:47.715658 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:02:47.715658 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:02:47.715658 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 587/1156] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 15:02:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:02:47.720771 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e42fcb-99b5-47fa-9ff1-cde1d01f2656 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:47.820177 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e42fcb-99b5-47fa-9ff1-cde1d01f2656 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:47.855485 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccc2577-a4c1-4a3a-83d4-8e26061071d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:47.856652 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc2577-a4c1-4a3a-83d4-8e26061071d1 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:47.857043 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc2577-a4c1-4a3a-83d4-8e26061071d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:47.857324 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc2577-a4c1-4a3a-83d4-8e26061071d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:47.858065 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccc2577-a4c1-4a3a-83d4-8e26061071d1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:47.858373 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccc2577-a4c1-4a3a-83d4-8e26061071d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:47.859761 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccc2577-a4c1-4a3a-83d4-8e26061071d1 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:47.861402 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:47.869532 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 570/1157] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:02:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:47.876328 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9c6776-d6c4-4827-8a65-5d607dca3697 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:47.965872 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:47.973499 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9c6776-d6c4-4827-8a65-5d607dca3697 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:48.006099 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a506ba89-4b6b-4e4f-ae99-e6f8e3488312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:48.007000 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a506ba89-4b6b-4e4f-ae99-e6f8e3488312 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:48.007285 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a506ba89-4b6b-4e4f-ae99-e6f8e3488312 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:48.007560 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a506ba89-4b6b-4e4f-ae99-e6f8e3488312 None admin] PATH_INFO: `/v3/users/1dbe094c671c439e8e87024b53e5b125` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:48.008725 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a506ba89-4b6b-4e4f-ae99-e6f8e3488312 None admin] RBAC: Authorizing `identity:get_user(user_id=1dbe094c671c439e8e87024b53e5b125)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:48.010149 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a506ba89-4b6b-4e4f-ae99-e6f8e3488312 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:48.011814 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 588/1158] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:02:47 2022] GET /identity/v3/users/1dbe094c671c439e8e87024b53e5b125 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:48.017584 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccc2577-a4c1-4a3a-83d4-8e26061071d1 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:48.112767 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccc2577-a4c1-4a3a-83d4-8e26061071d1 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:48.144793 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ad20aa-6083-4baf-a908-9f49b1b75bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:48.145420 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad20aa-6083-4baf-a908-9f49b1b75bde None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:48.145703 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad20aa-6083-4baf-a908-9f49b1b75bde None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:48.145979 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ad20aa-6083-4baf-a908-9f49b1b75bde None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:48.205652 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:48.212493 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ad20aa-6083-4baf-a908-9f49b1b75bde None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:48.214190 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ad20aa-6083-4baf-a908-9f49b1b75bde None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:48.309091 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 571/1159] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:02:48 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 15:02:48.315072 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a506ba89-4b6b-4e4f-ae99-e6f8e3488312 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:48.414160 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a506ba89-4b6b-4e4f-ae99-e6f8e3488312 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:48.446719 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8385e1-1c97-4190-bde9-06b129491ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:48.447599 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8385e1-1c97-4190-bde9-06b129491ebf None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:48.447886 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8385e1-1c97-4190-bde9-06b129491ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:48.448160 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8385e1-1c97-4190-bde9-06b129491ebf None admin] PATH_INFO: `/v3/projects/b82f749e9dec4deeb6a308655841e0ee` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:48.449336 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8385e1-1c97-4190-bde9-06b129491ebf None admin] RBAC: Authorizing `identity:get_project(project_id=b82f749e9dec4deeb6a308655841e0ee)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:48.450791 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8385e1-1c97-4190-bde9-06b129491ebf None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:48.452515 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 589/1160] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:02:48 2022] GET /identity/v3/projects/b82f749e9dec4deeb6a308655841e0ee => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:48.458578 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ad20aa-6083-4baf-a908-9f49b1b75bde None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:48.553373 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:48.559715 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ad20aa-6083-4baf-a908-9f49b1b75bde None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:48.594190 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc2486d-2f52-4027-aeeb-1f2660be51cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:48.594833 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc2486d-2f52-4027-aeeb-1f2660be51cd None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:48.595142 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc2486d-2f52-4027-aeeb-1f2660be51cd None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:48.595422 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc2486d-2f52-4027-aeeb-1f2660be51cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:48.596238 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc2486d-2f52-4027-aeeb-1f2660be51cd None admin] RBAC: Adding query filter params (role.id=759d845eb6284e42931b4265af448ebc, scope.project.id=b82f749e9dec4deeb6a308655841e0ee, user.id=1dbe094c671c439e8e87024b53e5b125) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:48.596549 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc2486d-2f52-4027-aeeb-1f2660be51cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:48.597963 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc2486d-2f52-4027-aeeb-1f2660be51cd None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:48.603661 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 572/1161] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:02:48 2022] GET /identity/v3/role_assignments?user.id=1dbe094c671c439e8e87024b53e5b125&scope.project.id=b82f749e9dec4deeb6a308655841e0ee&role.id=759d845eb6284e42931b4265af448ebc => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:50.861537 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d7843f-14b9-4a07-af2e-d9d005f4e8ca None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:50.862074 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d7843f-14b9-4a07-af2e-d9d005f4e8ca None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:50.862170 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d7843f-14b9-4a07-af2e-d9d005f4e8ca None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:50.863536 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 590/1162] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:02:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:02:50.869202 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8aef3-8794-40df-a7ea-6fa7d6a375c2 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:50.869644 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8aef3-8794-40df-a7ea-6fa7d6a375c2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:50.869762 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8aef3-8794-40df-a7ea-6fa7d6a375c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:50.886201 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5fb8aef3-8794-40df-a7ea-6fa7d6a375c2 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:02:50.888290 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5fb8aef3-8794-40df-a7ea-6fa7d6a375c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:02:50.961042 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 573/1163] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:02:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:02:50.983604 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d7843f-14b9-4a07-af2e-d9d005f4e8ca None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:51.013906 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:51.082927 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d7843f-14b9-4a07-af2e-d9d005f4e8ca None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:51.115716 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eebf331-5380-4e9b-851c-b55835048b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:51.116286 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebf331-5380-4e9b-851c-b55835048b10 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:51.116809 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebf331-5380-4e9b-851c-b55835048b10 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:51.117088 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eebf331-5380-4e9b-851c-b55835048b10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:51.181153 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eebf331-5380-4e9b-851c-b55835048b10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:51.182590 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eebf331-5380-4e9b-851c-b55835048b10 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:51.211001 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:51.281392 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 591/1164] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:02:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 15:02:51.287638 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8aef3-8794-40df-a7ea-6fa7d6a375c2 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:51.378425 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:51.388807 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8aef3-8794-40df-a7ea-6fa7d6a375c2 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:51.421068 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3170c5d4-5fa9-42c6-9e65-953562d4f65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:51.421857 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170c5d4-5fa9-42c6-9e65-953562d4f65e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:51.422134 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170c5d4-5fa9-42c6-9e65-953562d4f65e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:51.422406 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170c5d4-5fa9-42c6-9e65-953562d4f65e None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:51.429066 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3170c5d4-5fa9-42c6-9e65-953562d4f65e None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:51.430489 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3170c5d4-5fa9-42c6-9e65-953562d4f65e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:51.444608 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-3170c5d4-5fa9-42c6-9e65-953562d4f65e None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 27 15:02:51.444608 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:51.444608 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 27 15:02:51.444608 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 27 15:02:51.444608 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 15:02:51.444608 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 15:02:51.444608 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 15:02:51.444608 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 15:02:51.444608 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 27 15:02:51.444608 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:02:51.444608 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 15:02:51.444608 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:02:51.444608 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:02:51.444608 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:02:51.444608 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:02:51.444608 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:02:51.444608 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:02:51.445817 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:02:51.445817 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:02:51.445817 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:02:51.445817 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:02:51.445817 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:02:51.445817 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:02:51.445817 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:02:51.445817 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:02:51.445817 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:02:51.445817 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:02:51.445817 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:02:51.445817 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:02:51.445817 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:02:51.445817 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:02:51.445817 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:02:51.445817 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:02:51.446874 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:02:51.446874 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:02:51.446874 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:02:51.446874 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:02:51.446874 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:02:51.446874 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:02:51.446874 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:02:51.446874 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:02:51.446874 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:02:51.446874 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:02:51.446874 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 27 15:02:51.446874 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:02:51.446874 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 27 15:02:51.446874 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:02:51.446874 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 574/1165] 10.222.0.50 () {56 vars in 1117 bytes} [Fri May 27 15:02:51 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:02:51.452557 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eebf331-5380-4e9b-851c-b55835048b10 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:51.548877 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eebf331-5380-4e9b-851c-b55835048b10 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:51.577425 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:51.583405 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a167d5e-5012-4497-b8bc-ad3e0c5fb7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:51.584051 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a167d5e-5012-4497-b8bc-ad3e0c5fb7ae None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:51.584329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a167d5e-5012-4497-b8bc-ad3e0c5fb7ae None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:51.584600 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a167d5e-5012-4497-b8bc-ad3e0c5fb7ae None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:51.585304 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a167d5e-5012-4497-b8bc-ad3e0c5fb7ae None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:02:51.585603 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a167d5e-5012-4497-b8bc-ad3e0c5fb7ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:51.586980 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a167d5e-5012-4497-b8bc-ad3e0c5fb7ae None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:51.594399 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 592/1166] 10.222.0.50 () {56 vars in 1115 bytes} [Fri May 27 15:02:51 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:53.792187 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81a484-3e3b-4aa9-b2e0-358c9e382a4f None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:53.792749 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81a484-3e3b-4aa9-b2e0-358c9e382a4f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:53.792749 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c81a484-3e3b-4aa9-b2e0-358c9e382a4f None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:53.794730 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 575/1167] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:02:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:02:53.799904 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef4198-a0eb-4dd0-8278-dc0d9bc44aa6 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:53.800305 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef4198-a0eb-4dd0-8278-dc0d9bc44aa6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:53.800467 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef4198-a0eb-4dd0-8278-dc0d9bc44aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:53.817694 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7bef4198-a0eb-4dd0-8278-dc0d9bc44aa6 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:02:53.818892 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7bef4198-a0eb-4dd0-8278-dc0d9bc44aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:02:53.883737 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 593/1168] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:02:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:02:53.906261 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c81a484-3e3b-4aa9-b2e0-358c9e382a4f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:54.006391 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c81a484-3e3b-4aa9-b2e0-358c9e382a4f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:54.040402 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7513d313-6385-4b27-b069-1950ac74c24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:54.041056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7513d313-6385-4b27-b069-1950ac74c24e None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:54.041336 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7513d313-6385-4b27-b069-1950ac74c24e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:54.041609 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7513d313-6385-4b27-b069-1950ac74c24e None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:54.042385 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7513d313-6385-4b27-b069-1950ac74c24e None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:54.044239 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7513d313-6385-4b27-b069-1950ac74c24e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:54.059085 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 576/1169] 10.222.0.50 () {60 vars in 1138 bytes} [Fri May 27 15:02:53 2022] POST /identity/v3/domains => generated 267 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:02:56.849952 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7784e-75f1-44fd-9945-1567b3059b8d None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:56.850522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7784e-75f1-44fd-9945-1567b3059b8d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:56.850794 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7784e-75f1-44fd-9945-1567b3059b8d None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:56.852882 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 594/1170] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:02:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:02:56.859915 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a7d121-1df1-47d5-b2a3-92a0ce793b23 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:56.860379 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a7d121-1df1-47d5-b2a3-92a0ce793b23 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:56.860717 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a7d121-1df1-47d5-b2a3-92a0ce793b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:56.881040 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-22a7d121-1df1-47d5-b2a3-92a0ce793b23 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:02:56.882549 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-22a7d121-1df1-47d5-b2a3-92a0ce793b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:02:56.886273 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:56.952557 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 577/1171] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:02:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:02:56.977302 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f7784e-75f1-44fd-9945-1567b3059b8d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:57.089268 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f7784e-75f1-44fd-9945-1567b3059b8d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:57.112973 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:57.123869 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cf673b-b62a-4e36-bc9a-e8667e2e9165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:57.124731 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf673b-b62a-4e36-bc9a-e8667e2e9165 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:57.125013 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf673b-b62a-4e36-bc9a-e8667e2e9165 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:57.125283 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf673b-b62a-4e36-bc9a-e8667e2e9165 None admin] PATH_INFO: `/v3/domains/5423a154b08b4dee9e96f791da6bf382` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:57.135687 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cf673b-b62a-4e36-bc9a-e8667e2e9165 None admin] RBAC: Authorizing `identity:get_domain(domain_id=5423a154b08b4dee9e96f791da6bf382)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:57.137818 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cf673b-b62a-4e36-bc9a-e8667e2e9165 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:57.139625 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 595/1172] 10.222.0.50 () {56 vars in 1183 bytes} [Fri May 27 15:02:56 2022] GET /identity/v3/domains/5423a154b08b4dee9e96f791da6bf382 => generated 267 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:02:57.145427 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a7d121-1df1-47d5-b2a3-92a0ce793b23 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:02:57.244669 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a7d121-1df1-47d5-b2a3-92a0ce793b23 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:02:57.279311 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808e027d-72d2-4bc2-b932-e187974a4d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:02:57.279960 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e027d-72d2-4bc2-b932-e187974a4d75 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:57.280243 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e027d-72d2-4bc2-b932-e187974a4d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:57.280521 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808e027d-72d2-4bc2-b932-e187974a4d75 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:57.281233 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808e027d-72d2-4bc2-b932-e187974a4d75 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:02:57.282667 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808e027d-72d2-4bc2-b932-e187974a4d75 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:02:57.292224 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:02:57.309583 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 578/1173] 10.222.0.50 () {60 vars in 1141 bytes} [Fri May 27 15:02:57 2022] POST /identity/v3/projects => generated 361 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:02:59.845016 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149bf906-dd41-404b-b1eb-7aa430996d52 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:59.845581 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149bf906-dd41-404b-b1eb-7aa430996d52 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:59.845581 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149bf906-dd41-404b-b1eb-7aa430996d52 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:59.846971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 596/1174] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:02:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:02:59.852918 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f10ad-6f5d-4063-802b-faf51a5f2cb9 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:02:59.853339 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f10ad-6f5d-4063-802b-faf51a5f2cb9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:02:59.853487 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f10ad-6f5d-4063-802b-faf51a5f2cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:02:59.872584 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f77f10ad-6f5d-4063-802b-faf51a5f2cb9 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:02:59.874151 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f77f10ad-6f5d-4063-802b-faf51a5f2cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:02:59.940962 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 579/1175] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:02:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:02:59.968600 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149bf906-dd41-404b-b1eb-7aa430996d52 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:03:00.113216 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149bf906-dd41-404b-b1eb-7aa430996d52 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:03:00.164420 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e8dfd-ec29-4c6d-81ac-beb6edde3e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:03:00.165957 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e8dfd-ec29-4c6d-81ac-beb6edde3e8e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:00.166479 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e8dfd-ec29-4c6d-81ac-beb6edde3e8e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:00.167052 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e8dfd-ec29-4c6d-81ac-beb6edde3e8e None admin] PATH_INFO: `/v3/domains/5423a154b08b4dee9e96f791da6bf382` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:00.169500 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e8dfd-ec29-4c6d-81ac-beb6edde3e8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=5423a154b08b4dee9e96f791da6bf382)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:03:00.171877 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e8dfd-ec29-4c6d-81ac-beb6edde3e8e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:03:00.175019 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 597/1176] 10.222.0.50 () {56 vars in 1183 bytes} [Fri May 27 15:02:59 2022] GET /identity/v3/domains/5423a154b08b4dee9e96f791da6bf382 => generated 267 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:03:00.181206 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77f10ad-6f5d-4063-802b-faf51a5f2cb9 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:03:00.277917 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77f10ad-6f5d-4063-802b-faf51a5f2cb9 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:03:00.302789 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:03:00.315961 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d9c1bd-cd4d-4f88-ac4c-cdf649e2ca30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:03:00.316626 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9c1bd-cd4d-4f88-ac4c-cdf649e2ca30 None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:00.316909 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9c1bd-cd4d-4f88-ac4c-cdf649e2ca30 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:00.317185 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9c1bd-cd4d-4f88-ac4c-cdf649e2ca30 None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:00.318145 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d9c1bd-cd4d-4f88-ac4c-cdf649e2ca30 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:03:00.320785 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d9c1bd-cd4d-4f88-ac4c-cdf649e2ca30 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:03:00.344075 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 580/1177] 10.222.0.50 () {60 vars in 1132 bytes} [Fri May 27 15:03:00 2022] POST /identity/v3/users => generated 311 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:03:03.017355 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f3ec23-74fc-4860-973a-770959ce5754 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:03.018051 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f3ec23-74fc-4860-973a-770959ce5754 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:03.018284 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f3ec23-74fc-4860-973a-770959ce5754 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:03.020518 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 598/1178] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:03:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:03:03.027005 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d15d5-c9ec-4532-8b73-69a9d572e36f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:03.027570 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d15d5-c9ec-4532-8b73-69a9d572e36f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:03.027880 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d15d5-c9ec-4532-8b73-69a9d572e36f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:03.046119 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-258d15d5-c9ec-4532-8b73-69a9d572e36f None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:03:03.047246 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-258d15d5-c9ec-4532-8b73-69a9d572e36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:03:03.144884 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 581/1179] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:03:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:03:03.179660 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f3ec23-74fc-4860-973a-770959ce5754 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:03:03.257710 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:03:03.310184 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f3ec23-74fc-4860-973a-770959ce5754 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:03:03.349645 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cce7f4f-305c-4f67-8d37-5c345bb99ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:03:03.350485 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce7f4f-305c-4f67-8d37-5c345bb99ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:03.350768 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce7f4f-305c-4f67-8d37-5c345bb99ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:03.351072 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce7f4f-305c-4f67-8d37-5c345bb99ef1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:03.361476 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cce7f4f-305c-4f67-8d37-5c345bb99ef1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:03:03.369062 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-0cce7f4f-305c-4f67-8d37-5c345bb99ef1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:03:03.369062 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:03:03.369062 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:03:03.369062 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:03:03.369062 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:03:03.369062 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:03:03.369062 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:03:03.369062 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:03:03.369062 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:03:03.369062 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:03:03.369062 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:03:03.369062 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:03:03.369062 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:03:03.369062 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:03:03.369062 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:03:03.369062 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:03:03.370182 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:03:03.370182 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:03:03.370182 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:03:03.370182 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:03:03.370182 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:03:03.370182 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:03:03.370182 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:03:03.370182 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:03:03.370182 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:03:03.370182 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:03:03.370182 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:03:03.370182 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:03:03.370182 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:03:03.370182 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:03:03.370182 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:03:03.370182 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:03:03.371154 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:03:03.371154 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:03:03.371154 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:03:03.371154 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:03:03.371154 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:03:03.371154 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:03:03.371154 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:03:03.371154 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:03:03.371154 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:03:03.371154 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:03:03.371154 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:03:03.371154 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:03:03.371154 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 599/1180] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 15:03:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:03:03.376292 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258d15d5-c9ec-4532-8b73-69a9d572e36f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:03:03.479057 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258d15d5-c9ec-4532-8b73-69a9d572e36f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:03:03.488866 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:03:03.514576 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d652ceb-d0a0-4d70-bbd7-a8d3cff13807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:03:03.515268 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d652ceb-d0a0-4d70-bbd7-a8d3cff13807 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:03.515553 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d652ceb-d0a0-4d70-bbd7-a8d3cff13807 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:03.515828 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d652ceb-d0a0-4d70-bbd7-a8d3cff13807 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:03.516575 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d652ceb-d0a0-4d70-bbd7-a8d3cff13807 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:03:03.516882 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d652ceb-d0a0-4d70-bbd7-a8d3cff13807 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:03:03.518244 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d652ceb-d0a0-4d70-bbd7-a8d3cff13807 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:03:03.526130 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 582/1181] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:03:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:03:03.532686 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cce7f4f-305c-4f67-8d37-5c345bb99ef1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:03:03.628768 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cce7f4f-305c-4f67-8d37-5c345bb99ef1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:03:03.641509 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:03:03.662686 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6a99d-96f9-4c9a-8067-b91d0e63eba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:03:03.663568 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6a99d-96f9-4c9a-8067-b91d0e63eba4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:03.663862 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6a99d-96f9-4c9a-8067-b91d0e63eba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:03.664143 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6a99d-96f9-4c9a-8067-b91d0e63eba4 None admin] PATH_INFO: `/v3/users/4e54d64872a446cab3996fef40c53d40` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:03.677493 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb6a99d-96f9-4c9a-8067-b91d0e63eba4 None admin] RBAC: Authorizing `identity:get_user(user_id=4e54d64872a446cab3996fef40c53d40)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:03:03.678957 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb6a99d-96f9-4c9a-8067-b91d0e63eba4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:03:03.680595 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 600/1182] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:03:03 2022] GET /identity/v3/users/4e54d64872a446cab3996fef40c53d40 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:03:03.686370 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d652ceb-d0a0-4d70-bbd7-a8d3cff13807 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:03:03.783898 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d652ceb-d0a0-4d70-bbd7-a8d3cff13807 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:03:03.823341 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e9f01-31af-4157-add5-f4964eb19bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:03:03.824487 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e9f01-31af-4157-add5-f4964eb19bb7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:03.824785 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e9f01-31af-4157-add5-f4964eb19bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:03.825058 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e9f01-31af-4157-add5-f4964eb19bb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:03.863301 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:03:03.891851 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9e9f01-31af-4157-add5-f4964eb19bb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:03:03.893231 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9e9f01-31af-4157-add5-f4964eb19bb7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:03:03.986614 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 583/1183] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:03:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 15:03:03.992520 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6a99d-96f9-4c9a-8067-b91d0e63eba4 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:03:04.090643 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6a99d-96f9-4c9a-8067-b91d0e63eba4 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:03:04.125904 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0befeffa-33ae-48e8-ab47-06b2ca8eee40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:03:04.126756 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0befeffa-33ae-48e8-ab47-06b2ca8eee40 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:04.127066 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0befeffa-33ae-48e8-ab47-06b2ca8eee40 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:04.127344 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0befeffa-33ae-48e8-ab47-06b2ca8eee40 None admin] PATH_INFO: `/v3/projects/0cc202961a8b43d39c1b40bbc3eab885` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:04.128516 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0befeffa-33ae-48e8-ab47-06b2ca8eee40 None admin] RBAC: Authorizing `identity:get_project(project_id=0cc202961a8b43d39c1b40bbc3eab885)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:03:04.129936 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0befeffa-33ae-48e8-ab47-06b2ca8eee40 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:03:04.131670 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 601/1184] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:03:03 2022] GET /identity/v3/projects/0cc202961a8b43d39c1b40bbc3eab885 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:03:04.138393 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e9f01-31af-4157-add5-f4964eb19bb7 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:03:04.204434 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:03:04.234625 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e9f01-31af-4157-add5-f4964eb19bb7 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:03:04.267193 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2db6ae-576c-4557-a06f-3ec66e531fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:03:04.267835 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2db6ae-576c-4557-a06f-3ec66e531fb7 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:04.268118 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2db6ae-576c-4557-a06f-3ec66e531fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:04.268393 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2db6ae-576c-4557-a06f-3ec66e531fb7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:04.269358 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2db6ae-576c-4557-a06f-3ec66e531fb7 None admin] RBAC: Adding query filter params (role.id=759d845eb6284e42931b4265af448ebc, scope.project.id=0cc202961a8b43d39c1b40bbc3eab885, user.id=4e54d64872a446cab3996fef40c53d40) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:03:04.269782 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2db6ae-576c-4557-a06f-3ec66e531fb7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:03:04.271223 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2db6ae-576c-4557-a06f-3ec66e531fb7 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:03:04.276487 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 584/1185] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:03:04 2022] GET /identity/v3/role_assignments?user.id=4e54d64872a446cab3996fef40c53d40&scope.project.id=0cc202961a8b43d39c1b40bbc3eab885&role.id=759d845eb6284e42931b4265af448ebc => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:03:06.534034 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824dd1c1-7797-4e35-a5c5-07dc1224f33e None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:06.534612 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824dd1c1-7797-4e35-a5c5-07dc1224f33e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:06.534714 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824dd1c1-7797-4e35-a5c5-07dc1224f33e None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:06.536240 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 602/1186] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:03:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:03:06.541572 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8744f6e2-f2c0-4f79-9f1d-75d7cc8ec330 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:06.542016 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8744f6e2-f2c0-4f79-9f1d-75d7cc8ec330 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:06.542141 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8744f6e2-f2c0-4f79-9f1d-75d7cc8ec330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:06.559289 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8744f6e2-f2c0-4f79-9f1d-75d7cc8ec330 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:03:06.560470 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8744f6e2-f2c0-4f79-9f1d-75d7cc8ec330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:03:06.625601 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 585/1187] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:03:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:03:06.651237 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824dd1c1-7797-4e35-a5c5-07dc1224f33e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:03:06.693211 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:03:06.755004 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824dd1c1-7797-4e35-a5c5-07dc1224f33e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:03:06.797347 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572f1c9-73d0-4c21-acf6-3250660cf73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:03:06.798061 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572f1c9-73d0-4c21-acf6-3250660cf73b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:06.798366 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572f1c9-73d0-4c21-acf6-3250660cf73b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:06.798627 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5572f1c9-73d0-4c21-acf6-3250660cf73b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:06.813562 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5572f1c9-73d0-4c21-acf6-3250660cf73b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:03:06.822273 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-5572f1c9-73d0-4c21-acf6-3250660cf73b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:03:06.822273 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:03:06.822273 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:03:06.822273 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:03:06.822273 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:03:06.822273 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:03:06.822273 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:03:06.822273 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:03:06.822273 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:03:06.822273 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:03:06.822273 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:03:06.822273 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:03:06.822273 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:03:06.822273 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:03:06.822273 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:03:06.822273 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:03:06.825073 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:03:06.825073 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:03:06.825073 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:03:06.825073 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:03:06.825073 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:03:06.825073 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:03:06.825073 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:03:06.825073 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:03:06.825073 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:03:06.825073 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:03:06.825073 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:03:06.825073 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:03:06.825073 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:03:06.825073 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:03:06.825073 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:03:06.825073 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:03:06.827031 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:03:06.827031 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:03:06.827031 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:03:06.827031 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:03:06.827031 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:03:06.827031 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:03:06.827031 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:03:06.827031 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:03:06.827031 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:03:06.827031 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:03:06.827031 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:03:06.827031 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:03:06.827031 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 603/1188] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 15:03:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:03:06.830992 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8744f6e2-f2c0-4f79-9f1d-75d7cc8ec330 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:03:06.900393 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:03:06.945184 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8744f6e2-f2c0-4f79-9f1d-75d7cc8ec330 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:03:06.984499 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b73b8-163f-466a-a621-a79a9d9843f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:03:06.985292 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b73b8-163f-466a-a621-a79a9d9843f4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:06.985642 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b73b8-163f-466a-a621-a79a9d9843f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:06.985982 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9b73b8-163f-466a-a621-a79a9d9843f4 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:06.986933 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9b73b8-163f-466a-a621-a79a9d9843f4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:03:06.987270 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9b73b8-163f-466a-a621-a79a9d9843f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:03:06.988666 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9b73b8-163f-466a-a621-a79a9d9843f4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:03:06.998759 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 586/1189] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:03:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:03:07.007328 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572f1c9-73d0-4c21-acf6-3250660cf73b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:03:07.122528 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:03:07.126071 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5572f1c9-73d0-4c21-acf6-3250660cf73b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:03:07.179019 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86255e90-116c-4f84-9ad9-73336f3948c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:03:07.179888 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86255e90-116c-4f84-9ad9-73336f3948c2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:07.180169 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86255e90-116c-4f84-9ad9-73336f3948c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:07.180441 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86255e90-116c-4f84-9ad9-73336f3948c2 None admin] PATH_INFO: `/v3/users/4e54d64872a446cab3996fef40c53d40` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:07.181651 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86255e90-116c-4f84-9ad9-73336f3948c2 None admin] RBAC: Authorizing `identity:get_user(user_id=4e54d64872a446cab3996fef40c53d40)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:03:07.183113 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86255e90-116c-4f84-9ad9-73336f3948c2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:03:07.185890 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 604/1190] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:03:07 2022] GET /identity/v3/users/4e54d64872a446cab3996fef40c53d40 => generated 311 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:03:07.191656 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b73b8-163f-466a-a621-a79a9d9843f4 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:03:07.341628 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9b73b8-163f-466a-a621-a79a9d9843f4 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:03:07.378970 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:03:07.389668 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf376d-7172-4622-8d14-737755c43a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:03:07.395843 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf376d-7172-4622-8d14-737755c43a26 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:07.396243 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf376d-7172-4622-8d14-737755c43a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:07.396615 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf376d-7172-4622-8d14-737755c43a26 None admin] PATH_INFO: `/v3/projects/0cc202961a8b43d39c1b40bbc3eab885` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:07.401282 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cf376d-7172-4622-8d14-737755c43a26 None admin] RBAC: Authorizing `identity:get_project(project_id=0cc202961a8b43d39c1b40bbc3eab885)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:03:07.403819 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cf376d-7172-4622-8d14-737755c43a26 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:03:07.405579 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 587/1191] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:03:07 2022] GET /identity/v3/projects/0cc202961a8b43d39c1b40bbc3eab885 => generated 361 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:03:07.419334 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86255e90-116c-4f84-9ad9-73336f3948c2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:03:07.552030 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86255e90-116c-4f84-9ad9-73336f3948c2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:03:07.624722 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1de39d-a23b-4ca3-b34f-62f3b94900a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:03:07.626985 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1de39d-a23b-4ca3-b34f-62f3b94900a3 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:07.627275 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1de39d-a23b-4ca3-b34f-62f3b94900a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:07.627555 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1de39d-a23b-4ca3-b34f-62f3b94900a3 None admin] PATH_INFO: `/v3/projects/0cc202961a8b43d39c1b40bbc3eab885/users/4e54d64872a446cab3996fef40c53d40/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:07.629744 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1de39d-a23b-4ca3-b34f-62f3b94900a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=0cc202961a8b43d39c1b40bbc3eab885, user_id=4e54d64872a446cab3996fef40c53d40, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:03:07.632017 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1de39d-a23b-4ca3-b34f-62f3b94900a3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:03:07.648014 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 605/1192] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 15:03:07 2022] PUT /identity/v3/projects/0cc202961a8b43d39c1b40bbc3eab885/users/4e54d64872a446cab3996fef40c53d40/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:03:09.935454 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d47b2f-e0e5-4073-859b-e9656efd5cf7 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:09.936187 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d47b2f-e0e5-4073-859b-e9656efd5cf7 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:09.936187 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d47b2f-e0e5-4073-859b-e9656efd5cf7 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:09.937560 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 588/1193] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:03:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:03:09.944777 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdab504-2206-4fea-9092-a3f8b054e492 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:09.945314 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdab504-2206-4fea-9092-a3f8b054e492 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:09.945683 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdab504-2206-4fea-9092-a3f8b054e492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:09.965166 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4fdab504-2206-4fea-9092-a3f8b054e492 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:03:09.966618 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4fdab504-2206-4fea-9092-a3f8b054e492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:03:09.989459 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:03:10.042156 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 606/1194] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:03:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:03:10.067143 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d47b2f-e0e5-4073-859b-e9656efd5cf7 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:03:10.183025 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d47b2f-e0e5-4073-859b-e9656efd5cf7 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:03:10.220910 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3ca687-829d-479c-a110-e702efe957ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:03:10.221693 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ca687-829d-479c-a110-e702efe957ae None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:10.222024 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ca687-829d-479c-a110-e702efe957ae None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:10.222346 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3ca687-829d-479c-a110-e702efe957ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:10.235112 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3ca687-829d-479c-a110-e702efe957ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:03:10.242834 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-9d3ca687-829d-479c-a110-e702efe957ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:03:10.242834 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:03:10.242834 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:03:10.242834 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:03:10.242834 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:03:10.242834 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:03:10.242834 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:03:10.242834 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:03:10.242834 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:03:10.242834 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:03:10.242834 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:03:10.242834 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:03:10.242834 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:03:10.242834 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:03:10.242834 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:03:10.242834 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:03:10.244725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:03:10.244725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:03:10.244725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:03:10.244725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:03:10.244725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:03:10.244725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:03:10.244725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:03:10.244725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:03:10.244725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:03:10.244725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:03:10.244725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:03:10.244725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:03:10.244725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:03:10.244725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:03:10.244725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:03:10.244725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:03:10.246555 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:03:10.246555 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:03:10.246555 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:03:10.246555 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:03:10.246555 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:03:10.246555 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:03:10.246555 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:03:10.246555 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:03:10.246555 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:03:10.246555 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:03:10.246555 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:03:10.246555 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:03:10.246555 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 589/1195] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 15:03:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:03:10.252246 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdab504-2206-4fea-9092-a3f8b054e492 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:03:10.363939 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdab504-2206-4fea-9092-a3f8b054e492 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:03:10.404900 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06cf5f9-77d4-47fe-899d-e8a34e876411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:03:10.405591 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06cf5f9-77d4-47fe-899d-e8a34e876411 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:10.405875 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06cf5f9-77d4-47fe-899d-e8a34e876411 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:10.406151 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06cf5f9-77d4-47fe-899d-e8a34e876411 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:10.406958 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06cf5f9-77d4-47fe-899d-e8a34e876411 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:03:10.407273 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06cf5f9-77d4-47fe-899d-e8a34e876411 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:03:10.408755 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06cf5f9-77d4-47fe-899d-e8a34e876411 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:03:10.418108 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 607/1196] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:03:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:03:10.424443 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3ca687-829d-479c-a110-e702efe957ae None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:03:10.485216 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:03:10.535382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3ca687-829d-479c-a110-e702efe957ae None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:03:10.570470 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09703ba-c8a3-4042-a933-e3b039451685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:03:10.571373 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09703ba-c8a3-4042-a933-e3b039451685 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:10.571658 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09703ba-c8a3-4042-a933-e3b039451685 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:10.571932 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09703ba-c8a3-4042-a933-e3b039451685 None admin] PATH_INFO: `/v3/users/4e54d64872a446cab3996fef40c53d40` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:10.573150 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09703ba-c8a3-4042-a933-e3b039451685 None admin] RBAC: Authorizing `identity:get_user(user_id=4e54d64872a446cab3996fef40c53d40)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:03:10.575276 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09703ba-c8a3-4042-a933-e3b039451685 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:03:10.576912 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 590/1197] 10.222.0.50 () {56 vars in 1177 bytes} [Fri May 27 15:03:10 2022] GET /identity/v3/users/4e54d64872a446cab3996fef40c53d40 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:03:10.582832 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06cf5f9-77d4-47fe-899d-e8a34e876411 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:03:10.602997 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:03:10.684317 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06cf5f9-77d4-47fe-899d-e8a34e876411 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:03:10.718832 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d333de5c-ecc4-4154-8fca-63eec86bb639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:03:10.719495 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d333de5c-ecc4-4154-8fca-63eec86bb639 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:10.719776 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d333de5c-ecc4-4154-8fca-63eec86bb639 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:10.720049 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d333de5c-ecc4-4154-8fca-63eec86bb639 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:10.792506 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d333de5c-ecc4-4154-8fca-63eec86bb639 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:03:10.794037 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d333de5c-ecc4-4154-8fca-63eec86bb639 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:03:10.812447 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:03:10.891926 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 608/1198] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:03:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 15:03:10.897818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09703ba-c8a3-4042-a933-e3b039451685 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:03:10.993975 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09703ba-c8a3-4042-a933-e3b039451685 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:03:11.016686 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:03:11.028214 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61127fb3-0daa-4251-93dd-bd31a5e02330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:03:11.029080 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61127fb3-0daa-4251-93dd-bd31a5e02330 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:11.029363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61127fb3-0daa-4251-93dd-bd31a5e02330 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:11.029639 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61127fb3-0daa-4251-93dd-bd31a5e02330 None admin] PATH_INFO: `/v3/projects/0cc202961a8b43d39c1b40bbc3eab885` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:11.030802 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61127fb3-0daa-4251-93dd-bd31a5e02330 None admin] RBAC: Authorizing `identity:get_project(project_id=0cc202961a8b43d39c1b40bbc3eab885)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:03:11.032251 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61127fb3-0daa-4251-93dd-bd31a5e02330 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:03:11.033938 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 591/1199] 10.222.0.50 () {56 vars in 1186 bytes} [Fri May 27 15:03:10 2022] GET /identity/v3/projects/0cc202961a8b43d39c1b40bbc3eab885 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:03:11.040051 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d333de5c-ecc4-4154-8fca-63eec86bb639 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:03:11.141653 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d333de5c-ecc4-4154-8fca-63eec86bb639 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:03:11.158477 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:03:11.175460 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0048528a-2574-45e1-bc54-cd3cba5535e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:03:11.176110 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0048528a-2574-45e1-bc54-cd3cba5535e4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:03:11.176390 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0048528a-2574-45e1-bc54-cd3cba5535e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:03:11.176665 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0048528a-2574-45e1-bc54-cd3cba5535e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:03:11.177489 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0048528a-2574-45e1-bc54-cd3cba5535e4 None admin] RBAC: Adding query filter params (role.id=759d845eb6284e42931b4265af448ebc, scope.project.id=0cc202961a8b43d39c1b40bbc3eab885, user.id=4e54d64872a446cab3996fef40c53d40) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:03:11.177799 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0048528a-2574-45e1-bc54-cd3cba5535e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:03:11.179251 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0048528a-2574-45e1-bc54-cd3cba5535e4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:03:11.185535 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 609/1200] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:03:11 2022] GET /identity/v3/role_assignments?user.id=4e54d64872a446cab3996fef40c53d40&scope.project.id=0cc202961a8b43d39c1b40bbc3eab885&role.id=759d845eb6284e42931b4265af448ebc => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:03:21.024528 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:03:21.191140 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=9 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:12.647838 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51cf8ac-6e0c-49cb-98fe-c70c1cc386a3 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:12.648461 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51cf8ac-6e0c-49cb-98fe-c70c1cc386a3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:12.648461 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51cf8ac-6e0c-49cb-98fe-c70c1cc386a3 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:12.654058 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 592/1201] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:04:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:04:12.655907 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729236db-69f5-4464-b74d-5464b0830ea0 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:12.656391 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729236db-69f5-4464-b74d-5464b0830ea0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:12.656513 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729236db-69f5-4464-b74d-5464b0830ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:12.679103 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-729236db-69f5-4464-b74d-5464b0830ea0 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:04:12.680298 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-729236db-69f5-4464-b74d-5464b0830ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:04:12.776500 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 610/1202] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:04:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 15:04:12.805577 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51cf8ac-6e0c-49cb-98fe-c70c1cc386a3 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:12.940356 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51cf8ac-6e0c-49cb-98fe-c70c1cc386a3 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:12.984315 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb26aa-79cf-4f1f-975b-9befc2510eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:12.985127 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb26aa-79cf-4f1f-975b-9befc2510eca None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:12.985425 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb26aa-79cf-4f1f-975b-9befc2510eca None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:12.985707 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fb26aa-79cf-4f1f-975b-9befc2510eca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:12.997897 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fb26aa-79cf-4f1f-975b-9befc2510eca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:12.999477 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fb26aa-79cf-4f1f-975b-9befc2510eca None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:13.026444 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-91fb26aa-79cf-4f1f-975b-9befc2510eca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:04:13.026444 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:04:13.026444 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:04:13.026444 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:04:13.026444 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:04:13.026444 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:04:13.026444 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:04:13.026444 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:04:13.026444 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:04:13.026444 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:04:13.026444 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:04:13.026444 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:04:13.026444 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:04:13.026444 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:04:13.026444 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:04:13.028231 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:04:13.028231 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:04:13.028231 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:04:13.028231 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:04:13.028231 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:04:13.028231 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:04:13.028231 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:04:13.028231 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:04:13.028231 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:04:13.028231 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:04:13.028231 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:04:13.028231 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:04:13.028231 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:04:13.028231 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:04:13.028231 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:04:13.028231 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:04:13.028231 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:04:13.029418 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:04:13.029418 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:04:13.029418 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:04:13.029418 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:04:13.029418 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 593/1203] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:04:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:04:13.036922 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729236db-69f5-4464-b74d-5464b0830ea0 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:13.169567 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729236db-69f5-4464-b74d-5464b0830ea0 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:13.218768 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:13.227501 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc5cde7-da0d-4bdd-8cca-178520edfde2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:13.228194 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5cde7-da0d-4bdd-8cca-178520edfde2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:13.228493 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5cde7-da0d-4bdd-8cca-178520edfde2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:13.229461 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc5cde7-da0d-4bdd-8cca-178520edfde2 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:13.230191 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc5cde7-da0d-4bdd-8cca-178520edfde2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:04:13.230496 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc5cde7-da0d-4bdd-8cca-178520edfde2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:13.231909 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc5cde7-da0d-4bdd-8cca-178520edfde2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:13.245097 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 611/1204] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:04:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:04:13.250967 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb26aa-79cf-4f1f-975b-9befc2510eca None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:13.324763 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:13.386388 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fb26aa-79cf-4f1f-975b-9befc2510eca None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:13.439690 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41f0840-a726-4aa5-ac76-6e0fb34c3def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:13.440390 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f0840-a726-4aa5-ac76-6e0fb34c3def None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:13.440680 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f0840-a726-4aa5-ac76-6e0fb34c3def None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:13.440977 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f0840-a726-4aa5-ac76-6e0fb34c3def None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:13.441849 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41f0840-a726-4aa5-ac76-6e0fb34c3def None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:13.443474 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41f0840-a726-4aa5-ac76-6e0fb34c3def None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:13.466145 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 594/1205] 10.222.0.50 () {60 vars in 1132 bytes} [Fri May 27 15:04:13 2022] POST /identity/v3/users => generated 251 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:04:16.276978 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af90eedf-04a9-46ac-b83e-43ea2dd51da2 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:16.277522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af90eedf-04a9-46ac-b83e-43ea2dd51da2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:16.277522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af90eedf-04a9-46ac-b83e-43ea2dd51da2 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:16.278949 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 612/1206] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:04:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:04:16.285331 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86dacd-4a6c-4978-bdca-b167a7858caa None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:16.285849 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86dacd-4a6c-4978-bdca-b167a7858caa None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:16.285950 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b86dacd-4a6c-4978-bdca-b167a7858caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:16.303670 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2b86dacd-4a6c-4978-bdca-b167a7858caa None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:04:16.304983 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2b86dacd-4a6c-4978-bdca-b167a7858caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:04:16.379789 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 595/1207] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:04:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:04:16.407719 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af90eedf-04a9-46ac-b83e-43ea2dd51da2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:16.539400 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af90eedf-04a9-46ac-b83e-43ea2dd51da2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:16.604602 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1927fb-9fbd-45fa-8f76-7a4bdebc86c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:16.605717 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1927fb-9fbd-45fa-8f76-7a4bdebc86c0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:16.606051 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1927fb-9fbd-45fa-8f76-7a4bdebc86c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:16.606465 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1927fb-9fbd-45fa-8f76-7a4bdebc86c0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:16.621591 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1927fb-9fbd-45fa-8f76-7a4bdebc86c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:04:16.631709 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-6e1927fb-9fbd-45fa-8f76-7a4bdebc86c0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 15:04:16.631709 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:04:16.631709 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:04:16.631709 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:04:16.631709 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:04:16.631709 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:04:16.631709 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:04:16.631709 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:04:16.631709 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:04:16.631709 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:04:16.631709 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:04:16.631709 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:04:16.631709 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:04:16.631709 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:04:16.631709 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:04:16.631709 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:04:16.632890 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:04:16.632890 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:04:16.632890 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:04:16.632890 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:04:16.632890 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:04:16.632890 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:04:16.632890 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:04:16.632890 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:04:16.632890 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:04:16.632890 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:04:16.632890 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:04:16.632890 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:04:16.632890 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:04:16.632890 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:04:16.632890 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:04:16.632890 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:04:16.633867 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:04:16.633867 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:04:16.633867 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:04:16.633867 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:04:16.633867 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:04:16.633867 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:04:16.633867 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:04:16.633867 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:04:16.633867 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:04:16.633867 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:04:16.633867 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 15:04:16.633867 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:04:16.633867 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 613/1208] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 15:04:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:04:16.640251 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b86dacd-4a6c-4978-bdca-b167a7858caa None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:16.695741 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:16.784845 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b86dacd-4a6c-4978-bdca-b167a7858caa None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:16.831844 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d01ad6-130b-4865-bd15-a34a70bd2ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:16.832730 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d01ad6-130b-4865-bd15-a34a70bd2ea5 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:16.833174 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d01ad6-130b-4865-bd15-a34a70bd2ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:16.833602 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d01ad6-130b-4865-bd15-a34a70bd2ea5 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:16.834630 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d01ad6-130b-4865-bd15-a34a70bd2ea5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:04:16.835101 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d01ad6-130b-4865-bd15-a34a70bd2ea5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:16.837007 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d01ad6-130b-4865-bd15-a34a70bd2ea5 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:16.852916 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 596/1209] 10.222.0.50 () {56 vars in 1103 bytes} [Fri May 27 15:04:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:04:16.862137 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1927fb-9fbd-45fa-8f76-7a4bdebc86c0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:16.943765 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:17.012593 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1927fb-9fbd-45fa-8f76-7a4bdebc86c0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:17.052538 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74735f2e-5f61-408a-b3d6-c49a26ae810b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:17.053546 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74735f2e-5f61-408a-b3d6-c49a26ae810b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:17.054022 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74735f2e-5f61-408a-b3d6-c49a26ae810b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:17.054378 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74735f2e-5f61-408a-b3d6-c49a26ae810b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:17.068677 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74735f2e-5f61-408a-b3d6-c49a26ae810b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:17.070374 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74735f2e-5f61-408a-b3d6-c49a26ae810b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:17.092395 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-74735f2e-5f61-408a-b3d6-c49a26ae810b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:04:17.092395 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:04:17.092395 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:04:17.092395 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:04:17.092395 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:04:17.092395 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:04:17.092395 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:04:17.092395 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:04:17.092395 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:04:17.092395 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:04:17.092395 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:04:17.092395 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:04:17.092395 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:04:17.092395 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:04:17.092395 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:04:17.094243 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:04:17.094243 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:04:17.094243 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:04:17.094243 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:04:17.094243 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:04:17.094243 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:04:17.094243 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:04:17.094243 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:04:17.094243 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:04:17.094243 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:04:17.094243 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:04:17.094243 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:04:17.094243 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:04:17.094243 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:04:17.094243 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:04:17.094243 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:04:17.094243 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:04:17.095338 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:04:17.095338 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:04:17.095338 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:04:17.095338 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:04:17.095338 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 614/1210] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:04:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:04:17.100095 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d01ad6-130b-4865-bd15-a34a70bd2ea5 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:17.213443 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:17.227826 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d01ad6-130b-4865-bd15-a34a70bd2ea5 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:17.272118 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a8f49a-96d1-4089-8a4e-124a19c7541e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:17.272880 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a8f49a-96d1-4089-8a4e-124a19c7541e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:17.273173 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a8f49a-96d1-4089-8a4e-124a19c7541e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:17.273494 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a8f49a-96d1-4089-8a4e-124a19c7541e None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:17.274270 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a8f49a-96d1-4089-8a4e-124a19c7541e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:04:17.274626 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a8f49a-96d1-4089-8a4e-124a19c7541e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:17.276801 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a8f49a-96d1-4089-8a4e-124a19c7541e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:17.289943 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 597/1211] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:04:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:04:17.296113 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74735f2e-5f61-408a-b3d6-c49a26ae810b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:17.411812 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74735f2e-5f61-408a-b3d6-c49a26ae810b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:17.413499 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:17.461466 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c197a1a0-b4bb-40a8-b819-ef3b32600538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:17.462334 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c197a1a0-b4bb-40a8-b819-ef3b32600538 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:17.462627 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c197a1a0-b4bb-40a8-b819-ef3b32600538 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:17.463515 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c197a1a0-b4bb-40a8-b819-ef3b32600538 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:17.472688 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c197a1a0-b4bb-40a8-b819-ef3b32600538 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:17.474338 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c197a1a0-b4bb-40a8-b819-ef3b32600538 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:17.490971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-c197a1a0-b4bb-40a8-b819-ef3b32600538 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 15:04:17.490971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:04:17.490971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:04:17.490971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:04:17.490971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:04:17.490971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:04:17.490971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:04:17.490971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:04:17.490971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:04:17.490971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:04:17.490971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:04:17.490971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:04:17.490971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 15:04:17.490971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 15:04:17.490971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 15:04:17.490971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 15:04:17.492070 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:04:17.492070 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:04:17.492070 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 15:04:17.492070 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:04:17.492070 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 15:04:17.492070 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:04:17.492070 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:04:17.492070 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:04:17.492070 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:04:17.492070 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:04:17.492070 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:04:17.492070 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:04:17.492070 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:04:17.492070 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:04:17.492070 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:04:17.492070 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:04:17.492070 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:04:17.492070 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:04:17.493061 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:04:17.493061 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:04:17.493061 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 15:04:17.493061 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 15:04:17.493061 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 15:04:17.493061 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 15:04:17.493061 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 15:04:17.493061 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 15:04:17.493061 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 15:04:17.493061 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:04:17.493061 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 615/1212] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:04:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:04:17.498605 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a8f49a-96d1-4089-8a4e-124a19c7541e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:17.624953 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a8f49a-96d1-4089-8a4e-124a19c7541e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:17.665136 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b480baad-32d7-45f8-834b-438267e1bd6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:17.665891 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480baad-32d7-45f8-834b-438267e1bd6b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:17.666933 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480baad-32d7-45f8-834b-438267e1bd6b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:17.667398 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b480baad-32d7-45f8-834b-438267e1bd6b None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:17.668237 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b480baad-32d7-45f8-834b-438267e1bd6b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:04:17.668569 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b480baad-32d7-45f8-834b-438267e1bd6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:17.670038 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b480baad-32d7-45f8-834b-438267e1bd6b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:17.689700 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 598/1213] 10.222.0.50 () {56 vars in 1143 bytes} [Fri May 27 15:04:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:04:17.695439 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c197a1a0-b4bb-40a8-b819-ef3b32600538 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:17.822279 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c197a1a0-b4bb-40a8-b819-ef3b32600538 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:17.861570 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f605f87b-79f1-41c9-907c-ffce22e07023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:17.862296 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605f87b-79f1-41c9-907c-ffce22e07023 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:17.862577 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605f87b-79f1-41c9-907c-ffce22e07023 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:17.862882 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f605f87b-79f1-41c9-907c-ffce22e07023 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:17.904464 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:17.945607 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f605f87b-79f1-41c9-907c-ffce22e07023 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:17.947646 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f605f87b-79f1-41c9-907c-ffce22e07023 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:18.066885 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 616/1214] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:04:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 15:04:18.067101 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b480baad-32d7-45f8-834b-438267e1bd6b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:18.086756 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:18.184493 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b480baad-32d7-45f8-834b-438267e1bd6b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:18.222498 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267191c9-7f4b-4e52-8617-4061958b585f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:18.223334 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267191c9-7f4b-4e52-8617-4061958b585f None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:18.223621 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267191c9-7f4b-4e52-8617-4061958b585f None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:18.223902 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267191c9-7f4b-4e52-8617-4061958b585f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:18.234549 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267191c9-7f4b-4e52-8617-4061958b585f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:18.236112 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267191c9-7f4b-4e52-8617-4061958b585f None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:18.255170 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-267191c9-7f4b-4e52-8617-4061958b585f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:04:18.255170 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:04:18.255170 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:04:18.255170 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:04:18.255170 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:04:18.255170 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:04:18.255170 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:04:18.255170 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:04:18.255170 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:04:18.255170 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:04:18.255170 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:04:18.255170 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:04:18.255170 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:04:18.255170 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:04:18.255170 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:04:18.256151 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:04:18.256151 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:04:18.256151 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:04:18.256151 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:04:18.256151 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:04:18.256151 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:04:18.256151 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:04:18.256151 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:04:18.256151 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:04:18.256151 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:04:18.256151 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:04:18.256151 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:04:18.256151 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:04:18.256151 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:04:18.256151 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:04:18.256151 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:04:18.256151 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:04:18.263110 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:04:18.263110 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:04:18.263110 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:04:18.263110 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:04:18.263110 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 599/1215] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:04:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:04:18.264610 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f605f87b-79f1-41c9-907c-ffce22e07023 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:18.332311 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:18.375954 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f605f87b-79f1-41c9-907c-ffce22e07023 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:18.411452 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf47195-bf59-428e-8277-9b120ef7b431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:18.412130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf47195-bf59-428e-8277-9b120ef7b431 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:18.412418 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf47195-bf59-428e-8277-9b120ef7b431 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:18.412699 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf47195-bf59-428e-8277-9b120ef7b431 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:18.413493 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf47195-bf59-428e-8277-9b120ef7b431 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:04:18.413818 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf47195-bf59-428e-8277-9b120ef7b431 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:18.415228 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf47195-bf59-428e-8277-9b120ef7b431 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:18.428332 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 617/1216] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:04:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:04:18.433935 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267191c9-7f4b-4e52-8617-4061958b585f None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:18.510447 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:18.550792 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267191c9-7f4b-4e52-8617-4061958b585f None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:18.589415 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15808459-535f-4af5-8367-228bf6c8d33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:18.590217 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15808459-535f-4af5-8367-228bf6c8d33e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:18.590498 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15808459-535f-4af5-8367-228bf6c8d33e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:18.590774 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15808459-535f-4af5-8367-228bf6c8d33e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:18.598067 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15808459-535f-4af5-8367-228bf6c8d33e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:18.599549 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15808459-535f-4af5-8367-228bf6c8d33e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:18.614652 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-15808459-535f-4af5-8367-228bf6c8d33e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:04:18.614652 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:04:18.614652 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:04:18.614652 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:04:18.614652 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:04:18.614652 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:04:18.614652 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:04:18.614652 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:04:18.614652 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:04:18.614652 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:04:18.614652 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:04:18.614652 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:04:18.614652 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 15:04:18.614652 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 15:04:18.614652 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 15:04:18.615802 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 15:04:18.615802 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:04:18.615802 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:04:18.615802 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:04:18.615802 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:04:18.615802 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:04:18.615802 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:04:18.615802 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:04:18.615802 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:04:18.615802 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:04:18.615802 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:04:18.615802 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:04:18.615802 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:04:18.615802 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:04:18.615802 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:04:18.615802 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:04:18.615802 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:04:18.616710 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 15:04:18.616710 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 15:04:18.616710 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 15:04:18.616710 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 15:04:18.616710 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 15:04:18.616710 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 15:04:18.616710 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:04:18.616710 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:04:18.616710 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 600/1217] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:04:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:04:18.621852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf47195-bf59-428e-8277-9b120ef7b431 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:18.735985 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf47195-bf59-428e-8277-9b120ef7b431 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:18.761437 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:18.774814 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12412c3d-ab8a-44a0-be3d-32c86597ee7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:18.776205 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12412c3d-ab8a-44a0-be3d-32c86597ee7e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:18.776515 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12412c3d-ab8a-44a0-be3d-32c86597ee7e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:18.776765 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12412c3d-ab8a-44a0-be3d-32c86597ee7e None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:18.777561 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12412c3d-ab8a-44a0-be3d-32c86597ee7e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:04:18.777872 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12412c3d-ab8a-44a0-be3d-32c86597ee7e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:18.779354 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12412c3d-ab8a-44a0-be3d-32c86597ee7e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:18.793459 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 618/1218] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 15:04:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:04:18.799505 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15808459-535f-4af5-8367-228bf6c8d33e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:18.917517 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15808459-535f-4af5-8367-228bf6c8d33e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:18.956707 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:18.964540 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be0bb1f-547c-43c3-8d3a-4fae10ae5d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:18.965430 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0bb1f-547c-43c3-8d3a-4fae10ae5d22 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:18.965717 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0bb1f-547c-43c3-8d3a-4fae10ae5d22 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:18.965998 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be0bb1f-547c-43c3-8d3a-4fae10ae5d22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:18.966835 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be0bb1f-547c-43c3-8d3a-4fae10ae5d22 None admin] RBAC: Adding query filter params (role.id=13f37584013544e7ad1e2c8f350da6d2, scope.project.id=daf12585428b41a2b45bdbc85413a6c4, user.id=8293dab3ffd647f5927d323572fdbdce) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:04:18.967189 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be0bb1f-547c-43c3-8d3a-4fae10ae5d22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:18.968643 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be0bb1f-547c-43c3-8d3a-4fae10ae5d22 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:18.974879 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 601/1219] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:04:18 2022] GET /identity/v3/role_assignments?user.id=8293dab3ffd647f5927d323572fdbdce&scope.project.id=daf12585428b41a2b45bdbc85413a6c4&role.id=13f37584013544e7ad1e2c8f350da6d2 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:04:21.450459 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb863f8a-7174-4030-b10b-9ce456f235cb None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:21.451012 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb863f8a-7174-4030-b10b-9ce456f235cb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:21.451012 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb863f8a-7174-4030-b10b-9ce456f235cb None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:21.452342 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 619/1220] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:04:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:04:21.458982 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3ccac0-02cd-4b21-9db7-c3aed82d9674 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:21.459480 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3ccac0-02cd-4b21-9db7-c3aed82d9674 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:21.459596 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3ccac0-02cd-4b21-9db7-c3aed82d9674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:21.479028 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ed3ccac0-02cd-4b21-9db7-c3aed82d9674 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:04:21.480430 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ed3ccac0-02cd-4b21-9db7-c3aed82d9674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:04:21.551121 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 602/1221] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:04:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:04:21.579683 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb863f8a-7174-4030-b10b-9ce456f235cb None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:21.685009 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb863f8a-7174-4030-b10b-9ce456f235cb None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:21.720348 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e047c1-7815-43ae-82b9-e841ee7e8312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:21.721153 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e047c1-7815-43ae-82b9-e841ee7e8312 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:21.721438 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e047c1-7815-43ae-82b9-e841ee7e8312 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:21.721706 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e047c1-7815-43ae-82b9-e841ee7e8312 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:21.732176 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e047c1-7815-43ae-82b9-e841ee7e8312 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:04:21.739924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-f4e047c1-7815-43ae-82b9-e841ee7e8312 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 15:04:21.739924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:04:21.739924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:04:21.739924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:04:21.739924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:04:21.739924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:04:21.739924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:04:21.739924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:04:21.739924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:04:21.739924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:04:21.739924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:04:21.739924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:04:21.739924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:04:21.739924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:04:21.739924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:04:21.739924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:04:21.740983 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:04:21.740983 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:04:21.740983 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:04:21.740983 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:04:21.740983 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:04:21.740983 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:04:21.740983 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:04:21.740983 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:04:21.740983 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:04:21.740983 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:04:21.740983 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:04:21.740983 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:04:21.740983 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:04:21.740983 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:04:21.740983 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:04:21.740983 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:04:21.741852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:04:21.741852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:04:21.741852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:04:21.741852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:04:21.741852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:04:21.741852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:04:21.741852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:04:21.741852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:04:21.741852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:04:21.741852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:04:21.741852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 15:04:21.741852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:04:21.741852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 620/1222] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 15:04:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:04:21.747815 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3ccac0-02cd-4b21-9db7-c3aed82d9674 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:21.847779 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3ccac0-02cd-4b21-9db7-c3aed82d9674 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:21.872572 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:21.886826 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf148574-f596-4214-9560-80791dcd2807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:21.887792 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf148574-f596-4214-9560-80791dcd2807 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:21.888311 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf148574-f596-4214-9560-80791dcd2807 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:21.888680 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf148574-f596-4214-9560-80791dcd2807 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:21.889794 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf148574-f596-4214-9560-80791dcd2807 None admin] RBAC: Adding query filter params (name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:04:21.890112 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf148574-f596-4214-9560-80791dcd2807 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:21.892139 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf148574-f596-4214-9560-80791dcd2807 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:21.910722 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 603/1223] 10.222.0.50 () {56 vars in 1103 bytes} [Fri May 27 15:04:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:04:21.912025 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e047c1-7815-43ae-82b9-e841ee7e8312 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:21.959044 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:22.024019 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e047c1-7815-43ae-82b9-e841ee7e8312 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:22.060778 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f399432-e959-4192-8bcc-57ac698e5567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:22.061569 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f399432-e959-4192-8bcc-57ac698e5567 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:22.061855 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f399432-e959-4192-8bcc-57ac698e5567 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:22.062132 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f399432-e959-4192-8bcc-57ac698e5567 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:22.074101 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f399432-e959-4192-8bcc-57ac698e5567 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:22.076021 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f399432-e959-4192-8bcc-57ac698e5567 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:22.096520 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-9f399432-e959-4192-8bcc-57ac698e5567 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:04:22.096520 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:04:22.096520 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:04:22.096520 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:04:22.096520 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:04:22.096520 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:04:22.096520 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:04:22.096520 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:04:22.096520 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:04:22.096520 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:04:22.096520 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:04:22.096520 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:04:22.096520 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:04:22.096520 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:04:22.096520 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:04:22.097522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:04:22.097522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:04:22.097522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:04:22.097522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:04:22.097522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:04:22.097522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:04:22.097522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:04:22.097522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:04:22.097522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:04:22.097522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:04:22.097522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:04:22.097522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:04:22.097522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:04:22.097522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:04:22.097522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:04:22.097522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:04:22.097522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:04:22.098419 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:04:22.098419 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:04:22.098419 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:04:22.098419 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:04:22.098654 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 621/1224] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:04:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:04:22.104585 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf148574-f596-4214-9560-80791dcd2807 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:22.216140 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf148574-f596-4214-9560-80791dcd2807 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:22.254209 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e0953c-8a5c-445b-b05c-1467f10525d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:22.254892 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e0953c-8a5c-445b-b05c-1467f10525d3 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:22.255179 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e0953c-8a5c-445b-b05c-1467f10525d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:22.255452 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e0953c-8a5c-445b-b05c-1467f10525d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:22.256174 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e0953c-8a5c-445b-b05c-1467f10525d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:04:22.256484 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e0953c-8a5c-445b-b05c-1467f10525d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:22.257851 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e0953c-8a5c-445b-b05c-1467f10525d3 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:22.272254 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 604/1225] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:04:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:04:22.279013 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f399432-e959-4192-8bcc-57ac698e5567 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:22.392920 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:22.401311 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f399432-e959-4192-8bcc-57ac698e5567 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:22.435619 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7d8c2-b21a-48b7-a247-ef64a2c0aa81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:22.436418 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7d8c2-b21a-48b7-a247-ef64a2c0aa81 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:22.436701 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7d8c2-b21a-48b7-a247-ef64a2c0aa81 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:22.436976 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7d8c2-b21a-48b7-a247-ef64a2c0aa81 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:22.444622 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c7d8c2-b21a-48b7-a247-ef64a2c0aa81 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:22.446054 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c7d8c2-b21a-48b7-a247-ef64a2c0aa81 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:22.461602 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-17c7d8c2-b21a-48b7-a247-ef64a2c0aa81 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 15:04:22.461602 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:04:22.461602 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:04:22.461602 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:04:22.461602 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:04:22.461602 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:04:22.461602 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:04:22.461602 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:04:22.461602 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:04:22.461602 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:04:22.461602 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:04:22.461602 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:04:22.461602 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 15:04:22.461602 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 15:04:22.461602 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 15:04:22.461602 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 15:04:22.464226 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:04:22.464226 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:04:22.464226 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 15:04:22.464226 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:04:22.464226 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 15:04:22.464226 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:04:22.464226 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:04:22.464226 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:04:22.464226 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:04:22.464226 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:04:22.464226 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:04:22.464226 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:04:22.464226 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:04:22.464226 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:04:22.464226 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:04:22.464226 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:04:22.464226 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:04:22.464226 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:04:22.465391 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:04:22.465391 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:04:22.465391 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 15:04:22.465391 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 15:04:22.465391 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 15:04:22.465391 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 15:04:22.465391 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 15:04:22.465391 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 15:04:22.465391 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 15:04:22.465391 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:04:22.465391 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 622/1226] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:04:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:04:22.470036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e0953c-8a5c-445b-b05c-1467f10525d3 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:22.527079 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:22.585287 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e0953c-8a5c-445b-b05c-1467f10525d3 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:22.625688 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d4b1e6-98f1-4a23-bd5b-212f310bf17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:22.626359 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d4b1e6-98f1-4a23-bd5b-212f310bf17e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:22.626645 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d4b1e6-98f1-4a23-bd5b-212f310bf17e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:22.626963 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d4b1e6-98f1-4a23-bd5b-212f310bf17e None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:22.627798 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d4b1e6-98f1-4a23-bd5b-212f310bf17e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:04:22.628112 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d4b1e6-98f1-4a23-bd5b-212f310bf17e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:22.629749 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d4b1e6-98f1-4a23-bd5b-212f310bf17e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:22.670222 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 605/1227] 10.222.0.50 () {56 vars in 1143 bytes} [Fri May 27 15:04:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:04:22.671593 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7d8c2-b21a-48b7-a247-ef64a2c0aa81 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:22.782773 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7d8c2-b21a-48b7-a247-ef64a2c0aa81 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:22.821136 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6efda42-8cd6-4ff0-b618-6e5b08676a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:22.822800 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6efda42-8cd6-4ff0-b618-6e5b08676a5d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:22.823182 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6efda42-8cd6-4ff0-b618-6e5b08676a5d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:22.823457 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6efda42-8cd6-4ff0-b618-6e5b08676a5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:22.835051 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6efda42-8cd6-4ff0-b618-6e5b08676a5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:22.836522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6efda42-8cd6-4ff0-b618-6e5b08676a5d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:22.853432 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-f6efda42-8cd6-4ff0-b618-6e5b08676a5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:04:22.853432 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:04:22.853432 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:04:22.853432 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:04:22.853432 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:04:22.853432 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:04:22.853432 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:04:22.853432 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:04:22.853432 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:04:22.853432 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:04:22.853432 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:04:22.853432 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:04:22.853432 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:04:22.853432 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:04:22.853432 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:04:22.854642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:04:22.854642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:04:22.854642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:04:22.854642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:04:22.854642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:04:22.854642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:04:22.854642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:04:22.854642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:04:22.854642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:04:22.854642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:04:22.854642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:04:22.854642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:04:22.854642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:04:22.854642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:04:22.854642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:04:22.854642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:04:22.854642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:04:22.855671 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:04:22.855671 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:04:22.855671 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:04:22.855671 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:04:22.859190 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 623/1228] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:04:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:04:22.860284 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d4b1e6-98f1-4a23-bd5b-212f310bf17e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:22.983695 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d4b1e6-98f1-4a23-bd5b-212f310bf17e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:22.986894 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:23.027280 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b171671-9d1e-4b6a-bf19-a58a7027b8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:23.027947 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b171671-9d1e-4b6a-bf19-a58a7027b8dd None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:23.028236 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b171671-9d1e-4b6a-bf19-a58a7027b8dd None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:23.028513 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b171671-9d1e-4b6a-bf19-a58a7027b8dd None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:23.029278 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b171671-9d1e-4b6a-bf19-a58a7027b8dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:04:23.029538 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b171671-9d1e-4b6a-bf19-a58a7027b8dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:23.031000 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b171671-9d1e-4b6a-bf19-a58a7027b8dd None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:23.048861 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 606/1229] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:04:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:04:23.049999 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6efda42-8cd6-4ff0-b618-6e5b08676a5d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:23.060098 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:23.164875 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6efda42-8cd6-4ff0-b618-6e5b08676a5d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:23.204502 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02350be-0517-4db2-8832-a01fc7ef90b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:23.205291 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02350be-0517-4db2-8832-a01fc7ef90b7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:23.205573 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02350be-0517-4db2-8832-a01fc7ef90b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:23.205860 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02350be-0517-4db2-8832-a01fc7ef90b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:23.212682 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02350be-0517-4db2-8832-a01fc7ef90b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:23.214402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02350be-0517-4db2-8832-a01fc7ef90b7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:23.229550 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-e02350be-0517-4db2-8832-a01fc7ef90b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:04:23.229550 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:04:23.229550 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:04:23.229550 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:04:23.229550 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:04:23.229550 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:04:23.229550 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:04:23.229550 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:04:23.229550 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:04:23.229550 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:04:23.229550 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:04:23.229550 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:04:23.229550 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 15:04:23.229550 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 15:04:23.229550 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 15:04:23.230704 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 15:04:23.230704 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:04:23.230704 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:04:23.230704 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:04:23.230704 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:04:23.230704 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:04:23.230704 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:04:23.230704 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:04:23.230704 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:04:23.230704 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:04:23.230704 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:04:23.230704 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:04:23.230704 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:04:23.230704 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:04:23.230704 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:04:23.230704 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:04:23.230704 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:04:23.231783 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 15:04:23.231783 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 15:04:23.231783 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 15:04:23.231783 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 15:04:23.231783 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 15:04:23.231783 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 15:04:23.231783 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:04:23.231783 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:04:23.231783 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 624/1230] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:04:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:04:23.237438 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b171671-9d1e-4b6a-bf19-a58a7027b8dd None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:23.351575 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b171671-9d1e-4b6a-bf19-a58a7027b8dd None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:23.390615 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b60fe87-4b02-4cc3-b573-82915db52ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:23.391344 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b60fe87-4b02-4cc3-b573-82915db52ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:23.391631 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b60fe87-4b02-4cc3-b573-82915db52ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:23.391949 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b60fe87-4b02-4cc3-b573-82915db52ba4 None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:23.392733 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b60fe87-4b02-4cc3-b573-82915db52ba4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:04:23.393044 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b60fe87-4b02-4cc3-b573-82915db52ba4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:23.394508 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b60fe87-4b02-4cc3-b573-82915db52ba4 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:23.407481 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 607/1231] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 15:04:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:04:23.413318 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02350be-0517-4db2-8832-a01fc7ef90b7 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:23.485652 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:23.522238 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02350be-0517-4db2-8832-a01fc7ef90b7 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:23.557778 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb03d13a-3f4c-4b8c-98ec-61faf111a484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:23.558662 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03d13a-3f4c-4b8c-98ec-61faf111a484 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:23.558987 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03d13a-3f4c-4b8c-98ec-61faf111a484 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:23.559264 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03d13a-3f4c-4b8c-98ec-61faf111a484 None admin] PATH_INFO: `/v3/projects/daf12585428b41a2b45bdbc85413a6c4/users/8293dab3ffd647f5927d323572fdbdce/roles/13f37584013544e7ad1e2c8f350da6d2` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:23.574186 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb03d13a-3f4c-4b8c-98ec-61faf111a484 None admin] RBAC: Authorizing `identity:create_grant(project_id=daf12585428b41a2b45bdbc85413a6c4, user_id=8293dab3ffd647f5927d323572fdbdce, role_id=13f37584013544e7ad1e2c8f350da6d2)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:23.575703 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb03d13a-3f4c-4b8c-98ec-61faf111a484 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:23.589260 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 625/1232] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 15:04:23 2022] PUT /identity/v3/projects/daf12585428b41a2b45bdbc85413a6c4/users/8293dab3ffd647f5927d323572fdbdce/roles/13f37584013544e7ad1e2c8f350da6d2 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:04:25.955473 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7353654d-d846-447f-9196-855e2402f0ee None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:25.956208 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7353654d-d846-447f-9196-855e2402f0ee None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:25.956208 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7353654d-d846-447f-9196-855e2402f0ee None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:25.957690 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 608/1233] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:04:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:04:25.964267 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac43385-3ad4-4ade-ac06-284dccd722dd None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:25.964700 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac43385-3ad4-4ade-ac06-284dccd722dd None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:25.964812 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac43385-3ad4-4ade-ac06-284dccd722dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:25.984657 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-aac43385-3ad4-4ade-ac06-284dccd722dd None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:04:25.985870 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-aac43385-3ad4-4ade-ac06-284dccd722dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:04:26.059351 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 626/1234] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:04:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:04:26.084213 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7353654d-d846-447f-9196-855e2402f0ee None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:26.112741 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:26.207266 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7353654d-d846-447f-9196-855e2402f0ee None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:26.244392 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada60dd3-c70d-44eb-8488-36c58f93964c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:26.245190 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada60dd3-c70d-44eb-8488-36c58f93964c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:26.246838 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada60dd3-c70d-44eb-8488-36c58f93964c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:26.247154 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada60dd3-c70d-44eb-8488-36c58f93964c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:26.257752 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada60dd3-c70d-44eb-8488-36c58f93964c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:04:26.266411 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-ada60dd3-c70d-44eb-8488-36c58f93964c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 27 15:04:26.266411 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:04:26.266411 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:04:26.266411 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:04:26.266411 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:04:26.266411 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:04:26.266411 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:04:26.266411 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:04:26.266411 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:04:26.266411 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:04:26.266411 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:04:26.266411 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:04:26.266411 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:04:26.266411 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:04:26.266411 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:04:26.266411 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:04:26.267592 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:04:26.267592 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:04:26.267592 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:04:26.267592 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:04:26.267592 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:04:26.267592 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:04:26.267592 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:04:26.267592 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:04:26.267592 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:04:26.267592 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:04:26.267592 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:04:26.267592 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:04:26.267592 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:04:26.267592 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:04:26.267592 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:04:26.267592 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:04:26.268613 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:04:26.268613 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:04:26.268613 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:04:26.268613 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:04:26.268613 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:04:26.268613 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:04:26.268613 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:04:26.268613 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:04:26.268613 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:04:26.268613 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:04:26.268613 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 27 15:04:26.268613 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:04:26.272849 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 609/1235] 10.222.0.50 () {56 vars in 1102 bytes} [Fri May 27 15:04:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:04:26.273824 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac43385-3ad4-4ade-ac06-284dccd722dd None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:26.327092 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:26.387756 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac43385-3ad4-4ade-ac06-284dccd722dd None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:26.426873 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b95059-bf50-478b-b986-5dbc549ee286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:26.427659 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b95059-bf50-478b-b986-5dbc549ee286 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:26.428021 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b95059-bf50-478b-b986-5dbc549ee286 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:26.428379 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b95059-bf50-478b-b986-5dbc549ee286 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:26.429346 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b95059-bf50-478b-b986-5dbc549ee286 None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:04:26.429639 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b95059-bf50-478b-b986-5dbc549ee286 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:26.431882 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b95059-bf50-478b-b986-5dbc549ee286 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:26.441861 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 627/1236] 10.222.0.50 () {56 vars in 1103 bytes} [Fri May 27 15:04:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:04:26.448565 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada60dd3-c70d-44eb-8488-36c58f93964c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:26.547469 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:26.566092 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada60dd3-c70d-44eb-8488-36c58f93964c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:26.601346 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16494cd-3951-4bca-9afd-5ee05b6b1191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:26.602122 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16494cd-3951-4bca-9afd-5ee05b6b1191 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:26.602402 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16494cd-3951-4bca-9afd-5ee05b6b1191 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:26.602672 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16494cd-3951-4bca-9afd-5ee05b6b1191 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:26.614146 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16494cd-3951-4bca-9afd-5ee05b6b1191 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:26.615655 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16494cd-3951-4bca-9afd-5ee05b6b1191 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:26.633875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-f16494cd-3951-4bca-9afd-5ee05b6b1191 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:04:26.633875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:04:26.633875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:04:26.633875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:04:26.633875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:04:26.633875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:04:26.633875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:04:26.633875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:04:26.633875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:04:26.633875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:04:26.633875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:04:26.633875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:04:26.633875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:04:26.633875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:04:26.633875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:04:26.635768 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:04:26.635768 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:04:26.635768 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:04:26.635768 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:04:26.635768 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:04:26.635768 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:04:26.635768 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:04:26.635768 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:04:26.635768 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:04:26.635768 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:04:26.635768 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:04:26.635768 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:04:26.635768 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:04:26.635768 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:04:26.635768 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:04:26.635768 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:04:26.635768 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:04:26.636968 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:04:26.636968 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:04:26.636968 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:04:26.636968 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:04:26.636968 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 610/1237] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:04:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:04:26.641754 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b95059-bf50-478b-b986-5dbc549ee286 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:26.750969 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b95059-bf50-478b-b986-5dbc549ee286 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:26.762165 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:26.791174 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d2bfc-6087-4308-b320-0fa01b83607a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:26.792138 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d2bfc-6087-4308-b320-0fa01b83607a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:26.792429 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d2bfc-6087-4308-b320-0fa01b83607a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:26.792704 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d2bfc-6087-4308-b320-0fa01b83607a None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:26.793428 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355d2bfc-6087-4308-b320-0fa01b83607a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:04:26.793732 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355d2bfc-6087-4308-b320-0fa01b83607a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:26.795250 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355d2bfc-6087-4308-b320-0fa01b83607a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:26.809301 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 628/1238] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:04:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:04:26.815259 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16494cd-3951-4bca-9afd-5ee05b6b1191 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:26.935961 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16494cd-3951-4bca-9afd-5ee05b6b1191 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:26.973816 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dff9977-e295-4e67-89bf-876af61ce5e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:26.974624 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff9977-e295-4e67-89bf-876af61ce5e6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:26.974953 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff9977-e295-4e67-89bf-876af61ce5e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:26.975247 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dff9977-e295-4e67-89bf-876af61ce5e6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:26.977965 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:26.984494 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dff9977-e295-4e67-89bf-876af61ce5e6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:26.986451 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dff9977-e295-4e67-89bf-876af61ce5e6 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:27.003879 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-7dff9977-e295-4e67-89bf-876af61ce5e6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 15:04:27.003879 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:04:27.003879 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:04:27.003879 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:04:27.003879 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:04:27.003879 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:04:27.003879 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:04:27.003879 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:04:27.003879 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:04:27.003879 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:04:27.003879 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:04:27.003879 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:04:27.003879 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 15:04:27.003879 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 15:04:27.003879 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 15:04:27.003879 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 15:04:27.005027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:04:27.005027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:04:27.005027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 15:04:27.005027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:04:27.005027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 15:04:27.005027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:04:27.005027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:04:27.005027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:04:27.005027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:04:27.005027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:04:27.005027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:04:27.005027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:04:27.005027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:04:27.005027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:04:27.005027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:04:27.005027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:04:27.005027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:04:27.005027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:04:27.006194 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:04:27.006194 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:04:27.006194 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 15:04:27.006194 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 15:04:27.006194 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 15:04:27.006194 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 15:04:27.006194 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 15:04:27.006194 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 15:04:27.006194 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 15:04:27.006194 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:04:27.006194 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 611/1239] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:04:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:04:27.013550 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d2bfc-6087-4308-b320-0fa01b83607a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:27.128270 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d2bfc-6087-4308-b320-0fa01b83607a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:27.165031 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863eebf3-64f3-46ff-86f0-223859889077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:27.166652 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863eebf3-64f3-46ff-86f0-223859889077 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:27.167372 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863eebf3-64f3-46ff-86f0-223859889077 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:27.168000 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863eebf3-64f3-46ff-86f0-223859889077 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:27.169507 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863eebf3-64f3-46ff-86f0-223859889077 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:04:27.169931 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863eebf3-64f3-46ff-86f0-223859889077 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:27.172156 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863eebf3-64f3-46ff-86f0-223859889077 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:27.192564 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 629/1240] 10.222.0.50 () {56 vars in 1143 bytes} [Fri May 27 15:04:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:04:27.198992 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dff9977-e295-4e67-89bf-876af61ce5e6 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:27.302177 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dff9977-e295-4e67-89bf-876af61ce5e6 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:27.338094 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b7065-a566-43c9-a27a-16fa432639dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:27.338746 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b7065-a566-43c9-a27a-16fa432639dc None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:27.339332 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b7065-a566-43c9-a27a-16fa432639dc None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:27.339615 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b7065-a566-43c9-a27a-16fa432639dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:27.404720 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:27.412189 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79b7065-a566-43c9-a27a-16fa432639dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:27.413844 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79b7065-a566-43c9-a27a-16fa432639dc None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:27.524382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 612/1241] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:04:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 27 15:04:27.530789 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863eebf3-64f3-46ff-86f0-223859889077 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:27.560127 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:27.640282 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863eebf3-64f3-46ff-86f0-223859889077 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:27.677530 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4844806-8f03-4345-afcb-15ee216059da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:27.678251 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4844806-8f03-4345-afcb-15ee216059da None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:27.678534 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4844806-8f03-4345-afcb-15ee216059da None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:27.678808 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4844806-8f03-4345-afcb-15ee216059da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:27.689832 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4844806-8f03-4345-afcb-15ee216059da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:27.691335 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4844806-8f03-4345-afcb-15ee216059da None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:27.710556 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-a4844806-8f03-4345-afcb-15ee216059da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:04:27.710556 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:04:27.710556 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:04:27.710556 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:04:27.710556 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:04:27.710556 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:04:27.710556 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:04:27.710556 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:04:27.710556 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:04:27.710556 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:04:27.710556 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:04:27.710556 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:04:27.710556 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:04:27.710556 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:04:27.710556 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:04:27.711761 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:04:27.711761 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:04:27.711761 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:04:27.711761 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:04:27.711761 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:04:27.711761 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:04:27.711761 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:04:27.711761 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:04:27.711761 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:04:27.711761 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:04:27.711761 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:04:27.711761 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:04:27.711761 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:04:27.711761 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:04:27.711761 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:04:27.711761 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:04:27.711761 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:04:27.713642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:04:27.713642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:04:27.713642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:04:27.713642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:04:27.718897 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 630/1242] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:04:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:04:27.719056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b7065-a566-43c9-a27a-16fa432639dc None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:27.823917 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:27.830484 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b7065-a566-43c9-a27a-16fa432639dc None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:27.867600 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c2d75-d6e8-4c16-94a0-f96e57a7cde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:27.868289 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c2d75-d6e8-4c16-94a0-f96e57a7cde9 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:27.868642 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c2d75-d6e8-4c16-94a0-f96e57a7cde9 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:27.869093 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c2d75-d6e8-4c16-94a0-f96e57a7cde9 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:27.870023 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761c2d75-d6e8-4c16-94a0-f96e57a7cde9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:04:27.870378 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761c2d75-d6e8-4c16-94a0-f96e57a7cde9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:27.872020 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761c2d75-d6e8-4c16-94a0-f96e57a7cde9 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:27.886162 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 613/1243] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:04:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:04:27.892865 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4844806-8f03-4345-afcb-15ee216059da None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:27.985652 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:28.010628 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4844806-8f03-4345-afcb-15ee216059da None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:28.047028 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5372d114-d6ff-4aca-8d66-6d766eb7a9a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:28.047877 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372d114-d6ff-4aca-8d66-6d766eb7a9a6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:28.048207 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372d114-d6ff-4aca-8d66-6d766eb7a9a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:28.048487 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5372d114-d6ff-4aca-8d66-6d766eb7a9a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:28.055219 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5372d114-d6ff-4aca-8d66-6d766eb7a9a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:28.057185 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5372d114-d6ff-4aca-8d66-6d766eb7a9a6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:28.071218 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-5372d114-d6ff-4aca-8d66-6d766eb7a9a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:04:28.071218 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:04:28.071218 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:04:28.071218 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:04:28.071218 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:04:28.071218 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:04:28.071218 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:04:28.071218 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:04:28.071218 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:04:28.071218 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:04:28.071218 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:04:28.071218 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:04:28.071218 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 15:04:28.071218 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 15:04:28.071218 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 15:04:28.072924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 15:04:28.072924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:04:28.072924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:04:28.072924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:04:28.072924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:04:28.072924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:04:28.072924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:04:28.072924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:04:28.072924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:04:28.072924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:04:28.072924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:04:28.072924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:04:28.072924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:04:28.072924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:04:28.072924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:04:28.072924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:04:28.072924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:04:28.073930 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 15:04:28.073930 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 15:04:28.073930 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 15:04:28.073930 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 15:04:28.073930 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 15:04:28.073930 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 15:04:28.073930 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:04:28.073930 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:04:28.073930 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 631/1244] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:04:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:04:28.079671 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c2d75-d6e8-4c16-94a0-f96e57a7cde9 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:28.196296 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c2d75-d6e8-4c16-94a0-f96e57a7cde9 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:28.238389 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3602f0ca-520d-4488-a7cb-332b755baaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:28.239090 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602f0ca-520d-4488-a7cb-332b755baaef None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:28.239383 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602f0ca-520d-4488-a7cb-332b755baaef None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:28.239665 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602f0ca-520d-4488-a7cb-332b755baaef None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:28.240474 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3602f0ca-520d-4488-a7cb-332b755baaef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:04:28.240781 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3602f0ca-520d-4488-a7cb-332b755baaef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:28.242436 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3602f0ca-520d-4488-a7cb-332b755baaef None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:28.257882 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 614/1245] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 15:04:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:04:28.265130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5372d114-d6ff-4aca-8d66-6d766eb7a9a6 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:28.381226 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5372d114-d6ff-4aca-8d66-6d766eb7a9a6 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:28.417329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387090ea-d0db-4804-bb5b-e8519993491f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:28.417992 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387090ea-d0db-4804-bb5b-e8519993491f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:28.418278 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387090ea-d0db-4804-bb5b-e8519993491f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:28.418539 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387090ea-d0db-4804-bb5b-e8519993491f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:28.419391 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387090ea-d0db-4804-bb5b-e8519993491f None admin] RBAC: Adding query filter params (role.id=13f37584013544e7ad1e2c8f350da6d2, scope.project.id=daf12585428b41a2b45bdbc85413a6c4, user.id=8293dab3ffd647f5927d323572fdbdce) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:04:28.419696 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387090ea-d0db-4804-bb5b-e8519993491f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:28.421117 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387090ea-d0db-4804-bb5b-e8519993491f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:28.427001 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 632/1246] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:04:28 2022] GET /identity/v3/role_assignments?user.id=8293dab3ffd647f5927d323572fdbdce&scope.project.id=daf12585428b41a2b45bdbc85413a6c4&role.id=13f37584013544e7ad1e2c8f350da6d2 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:04:30.907836 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9a7e9-7784-46bb-bbf5-19ec68b5f4d9 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:30.908483 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9a7e9-7784-46bb-bbf5-19ec68b5f4d9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:30.908483 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed9a7e9-7784-46bb-bbf5-19ec68b5f4d9 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:30.909784 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 615/1247] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:04:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:04:30.916480 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c1ced-1f32-4d37-bf8b-5d2331b11a54 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:30.916944 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c1ced-1f32-4d37-bf8b-5d2331b11a54 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:30.917036 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419c1ced-1f32-4d37-bf8b-5d2331b11a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:30.923032 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:30.939110 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-419c1ced-1f32-4d37-bf8b-5d2331b11a54 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:04:30.940479 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-419c1ced-1f32-4d37-bf8b-5d2331b11a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:04:31.012251 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 633/1248] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:04:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:04:31.040345 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed9a7e9-7784-46bb-bbf5-19ec68b5f4d9 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:31.063507 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:31.165363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed9a7e9-7784-46bb-bbf5-19ec68b5f4d9 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:31.207092 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecadc8f6-d0bf-4ea9-a325-a6b4e75eaab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:31.207976 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecadc8f6-d0bf-4ea9-a325-a6b4e75eaab2 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:31.208300 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecadc8f6-d0bf-4ea9-a325-a6b4e75eaab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:31.208582 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecadc8f6-d0bf-4ea9-a325-a6b4e75eaab2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:31.221064 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecadc8f6-d0bf-4ea9-a325-a6b4e75eaab2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:04:31.230056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-ecadc8f6-d0bf-4ea9-a325-a6b4e75eaab2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:04:31.230056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:04:31.230056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:04:31.230056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:04:31.230056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:04:31.230056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:04:31.230056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:04:31.230056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:04:31.230056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:04:31.230056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:04:31.230056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:04:31.230056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:04:31.230056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:04:31.230056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:04:31.230056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:04:31.230056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:04:31.231363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:04:31.231363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:04:31.231363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:04:31.231363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:04:31.231363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:04:31.231363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:04:31.231363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:04:31.231363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:04:31.231363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:04:31.231363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:04:31.231363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:04:31.231363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:04:31.231363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:04:31.231363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:04:31.231363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:04:31.231363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:04:31.232328 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:04:31.232328 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:04:31.232328 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:04:31.232328 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:04:31.232328 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:04:31.232328 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:04:31.232328 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:04:31.232328 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:04:31.232328 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:04:31.232328 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:04:31.232328 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:04:31.232328 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:04:31.232328 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 616/1249] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 15:04:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:04:31.239407 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c1ced-1f32-4d37-bf8b-5d2331b11a54 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:31.350396 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419c1ced-1f32-4d37-bf8b-5d2331b11a54 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:31.383960 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:31.393006 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e1ab9a-fa20-4f86-bff0-13671a7c8c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:31.393688 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e1ab9a-fa20-4f86-bff0-13671a7c8c2f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:31.393973 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e1ab9a-fa20-4f86-bff0-13671a7c8c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:31.394253 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e1ab9a-fa20-4f86-bff0-13671a7c8c2f None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:31.395077 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e1ab9a-fa20-4f86-bff0-13671a7c8c2f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:04:31.395379 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e1ab9a-fa20-4f86-bff0-13671a7c8c2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:31.397096 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e1ab9a-fa20-4f86-bff0-13671a7c8c2f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:31.405893 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 634/1250] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:04:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:04:31.411665 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecadc8f6-d0bf-4ea9-a325-a6b4e75eaab2 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:31.495199 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:31.522638 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecadc8f6-d0bf-4ea9-a325-a6b4e75eaab2 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:31.562438 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b02c362-068d-4673-a9e7-d85ce68a3551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:31.563276 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02c362-068d-4673-a9e7-d85ce68a3551 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:31.563565 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02c362-068d-4673-a9e7-d85ce68a3551 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:31.563849 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02c362-068d-4673-a9e7-d85ce68a3551 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:31.574761 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b02c362-068d-4673-a9e7-d85ce68a3551 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:31.576240 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b02c362-068d-4673-a9e7-d85ce68a3551 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:31.594492 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-1b02c362-068d-4673-a9e7-d85ce68a3551 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:04:31.594492 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:04:31.594492 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:04:31.594492 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:04:31.594492 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:04:31.594492 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:04:31.594492 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:04:31.594492 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:04:31.594492 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:04:31.594492 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:04:31.594492 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:04:31.594492 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:04:31.594492 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:04:31.594492 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:04:31.594492 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:04:31.595553 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:04:31.595553 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:04:31.595553 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:04:31.595553 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:04:31.595553 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:04:31.595553 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:04:31.595553 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:04:31.595553 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:04:31.595553 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:04:31.595553 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:04:31.595553 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:04:31.595553 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:04:31.595553 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:04:31.595553 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:04:31.595553 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:04:31.595553 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:04:31.595553 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:04:31.596687 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:04:31.596687 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:04:31.596687 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:04:31.596687 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:04:31.596687 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 617/1251] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:04:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 27 15:04:31.603602 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e1ab9a-fa20-4f86-bff0-13671a7c8c2f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:31.709514 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e1ab9a-fa20-4f86-bff0-13671a7c8c2f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:31.746307 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db727f3b-8717-49c1-a17c-aa048f2b3cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:31.746999 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db727f3b-8717-49c1-a17c-aa048f2b3cce None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:31.747285 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db727f3b-8717-49c1-a17c-aa048f2b3cce None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:31.747564 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db727f3b-8717-49c1-a17c-aa048f2b3cce None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:31.748352 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db727f3b-8717-49c1-a17c-aa048f2b3cce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:04:31.748731 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db727f3b-8717-49c1-a17c-aa048f2b3cce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:31.750230 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db727f3b-8717-49c1-a17c-aa048f2b3cce None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:31.765536 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 635/1252] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:04:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:04:31.771267 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b02c362-068d-4673-a9e7-d85ce68a3551 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:31.885588 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b02c362-068d-4673-a9e7-d85ce68a3551 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:31.918033 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:31.927373 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c2e656-14ee-49da-902d-37534e330d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:31.928197 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c2e656-14ee-49da-902d-37534e330d7a None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:31.928479 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c2e656-14ee-49da-902d-37534e330d7a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:31.928754 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c2e656-14ee-49da-902d-37534e330d7a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:31.936775 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c2e656-14ee-49da-902d-37534e330d7a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:31.938227 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c2e656-14ee-49da-902d-37534e330d7a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:31.952929 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-41c2e656-14ee-49da-902d-37534e330d7a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 15:04:31.952929 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:04:31.952929 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:04:31.952929 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:04:31.952929 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:04:31.952929 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:04:31.952929 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:04:31.952929 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:04:31.952929 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:04:31.952929 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:04:31.952929 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:04:31.952929 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:04:31.952929 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 15:04:31.952929 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 15:04:31.952929 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 15:04:31.952929 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 15:04:31.954091 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:04:31.954091 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:04:31.954091 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 15:04:31.954091 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:04:31.954091 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 15:04:31.954091 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:04:31.954091 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:04:31.954091 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:04:31.954091 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:04:31.954091 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:04:31.954091 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:04:31.954091 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:04:31.954091 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:04:31.954091 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:04:31.954091 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:04:31.954091 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:04:31.954091 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:04:31.954091 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:04:31.959740 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:04:31.959740 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:04:31.959740 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 15:04:31.959740 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 15:04:31.959740 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 15:04:31.959740 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 15:04:31.959740 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 15:04:31.959740 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 15:04:31.959740 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 15:04:31.959740 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:04:31.959740 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 618/1253] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:04:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:04:31.960345 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db727f3b-8717-49c1-a17c-aa048f2b3cce None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:32.020207 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:32.068312 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db727f3b-8717-49c1-a17c-aa048f2b3cce None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:32.105462 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560fa306-7b73-4825-9998-ca0c41711063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:32.106136 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560fa306-7b73-4825-9998-ca0c41711063 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:32.106426 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560fa306-7b73-4825-9998-ca0c41711063 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:32.106706 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560fa306-7b73-4825-9998-ca0c41711063 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:32.107573 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560fa306-7b73-4825-9998-ca0c41711063 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:04:32.107884 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560fa306-7b73-4825-9998-ca0c41711063 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:32.109309 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560fa306-7b73-4825-9998-ca0c41711063 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:32.126950 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 636/1254] 10.222.0.50 () {56 vars in 1143 bytes} [Fri May 27 15:04:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:04:32.132850 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c2e656-14ee-49da-902d-37534e330d7a None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:32.244673 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c2e656-14ee-49da-902d-37534e330d7a None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:32.281240 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76733c4e-bfc6-4bce-a302-fb7ec4fa5774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:32.281817 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76733c4e-bfc6-4bce-a302-fb7ec4fa5774 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:32.282104 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76733c4e-bfc6-4bce-a302-fb7ec4fa5774 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:32.282386 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76733c4e-bfc6-4bce-a302-fb7ec4fa5774 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:32.341596 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:32.355282 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76733c4e-bfc6-4bce-a302-fb7ec4fa5774 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:32.356775 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76733c4e-bfc6-4bce-a302-fb7ec4fa5774 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:32.468286 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 619/1255] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:04:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 15:04:32.474356 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560fa306-7b73-4825-9998-ca0c41711063 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:32.578160 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560fa306-7b73-4825-9998-ca0c41711063 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:32.592905 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:32.615032 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16c3681-78ad-4e28-8f04-0d30d4fbb65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:32.615730 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c3681-78ad-4e28-8f04-0d30d4fbb65c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:32.616014 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c3681-78ad-4e28-8f04-0d30d4fbb65c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:32.616290 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c3681-78ad-4e28-8f04-0d30d4fbb65c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:32.628037 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16c3681-78ad-4e28-8f04-0d30d4fbb65c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:32.629517 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16c3681-78ad-4e28-8f04-0d30d4fbb65c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:32.648642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-b16c3681-78ad-4e28-8f04-0d30d4fbb65c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:04:32.648642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:04:32.648642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:04:32.648642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:04:32.648642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:04:32.648642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:04:32.648642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:04:32.648642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:04:32.648642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:04:32.648642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:04:32.648642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:04:32.648642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:04:32.648642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:04:32.648642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:04:32.648642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:04:32.649800 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:04:32.649800 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:04:32.649800 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:04:32.649800 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:04:32.649800 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:04:32.649800 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:04:32.649800 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:04:32.649800 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:04:32.649800 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:04:32.649800 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:04:32.649800 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:04:32.649800 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:04:32.649800 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:04:32.649800 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:04:32.649800 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:04:32.649800 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:04:32.649800 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:04:32.650803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:04:32.650803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:04:32.650803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:04:32.650803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:04:32.650803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 637/1256] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:04:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:04:32.656025 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76733c4e-bfc6-4bce-a302-fb7ec4fa5774 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:32.747565 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:32.760875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76733c4e-bfc6-4bce-a302-fb7ec4fa5774 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:32.796724 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8149c064-78b9-4888-b823-609924d0e3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:32.797385 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149c064-78b9-4888-b823-609924d0e3fc None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:32.797672 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149c064-78b9-4888-b823-609924d0e3fc None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:32.797942 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149c064-78b9-4888-b823-609924d0e3fc None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:32.798657 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8149c064-78b9-4888-b823-609924d0e3fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:04:32.798995 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8149c064-78b9-4888-b823-609924d0e3fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:32.800355 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8149c064-78b9-4888-b823-609924d0e3fc None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:32.813237 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 620/1257] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:04:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:04:32.818566 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16c3681-78ad-4e28-8f04-0d30d4fbb65c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:32.936867 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16c3681-78ad-4e28-8f04-0d30d4fbb65c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:32.970912 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d33e74-b1f4-496c-ae9f-a224e6f16104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:32.971707 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d33e74-b1f4-496c-ae9f-a224e6f16104 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:32.971990 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d33e74-b1f4-496c-ae9f-a224e6f16104 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:32.972263 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d33e74-b1f4-496c-ae9f-a224e6f16104 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:32.979376 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d33e74-b1f4-496c-ae9f-a224e6f16104 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:32.980836 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d33e74-b1f4-496c-ae9f-a224e6f16104 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:32.994340 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-55d33e74-b1f4-496c-ae9f-a224e6f16104 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:04:32.994340 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:04:32.994340 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:04:32.994340 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:04:32.994340 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:04:32.994340 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:04:32.994340 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:04:32.994340 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:04:32.994340 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:04:32.994340 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:04:32.994340 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:04:32.994340 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:04:32.994340 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 15:04:32.994340 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 15:04:32.994340 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 15:04:32.995902 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 15:04:32.995902 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:04:32.995902 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:04:32.995902 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:04:32.995902 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:04:32.995902 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:04:32.995902 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:04:32.995902 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:04:32.995902 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:04:32.995902 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:04:32.995902 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:04:32.995902 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:04:32.995902 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:04:32.995902 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:04:32.995902 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:04:32.995902 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:04:32.995902 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:04:32.997016 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 15:04:32.997016 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 15:04:32.997016 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 15:04:32.997016 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 15:04:32.997016 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 15:04:32.997016 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 15:04:32.997016 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:04:32.997016 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:04:32.997016 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 638/1258] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:04:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:04:33.001966 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8149c064-78b9-4888-b823-609924d0e3fc None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:33.097589 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8149c064-78b9-4888-b823-609924d0e3fc None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:33.130519 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eefecbd-6788-42a6-98bc-a6c20770d72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:33.131191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eefecbd-6788-42a6-98bc-a6c20770d72e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:33.131476 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eefecbd-6788-42a6-98bc-a6c20770d72e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:33.131752 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eefecbd-6788-42a6-98bc-a6c20770d72e None admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:33.132528 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eefecbd-6788-42a6-98bc-a6c20770d72e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:04:33.132835 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eefecbd-6788-42a6-98bc-a6c20770d72e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:33.134291 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eefecbd-6788-42a6-98bc-a6c20770d72e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:33.145948 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 621/1259] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 15:04:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:04:33.153040 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d33e74-b1f4-496c-ae9f-a224e6f16104 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:33.178715 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:33.252882 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d33e74-b1f4-496c-ae9f-a224e6f16104 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:33.285211 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516a0fbc-2f07-45c7-b9d9-03f9832af33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:33.285856 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a0fbc-2f07-45c7-b9d9-03f9832af33b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:33.286137 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a0fbc-2f07-45c7-b9d9-03f9832af33b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:33.286412 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516a0fbc-2f07-45c7-b9d9-03f9832af33b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:33.287256 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516a0fbc-2f07-45c7-b9d9-03f9832af33b None admin] RBAC: Adding query filter params (role.id=759d845eb6284e42931b4265af448ebc, scope.project.id=daf12585428b41a2b45bdbc85413a6c4, user.id=8293dab3ffd647f5927d323572fdbdce) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:04:33.287567 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516a0fbc-2f07-45c7-b9d9-03f9832af33b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:33.288982 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516a0fbc-2f07-45c7-b9d9-03f9832af33b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:33.294983 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 639/1260] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:04:33 2022] GET /identity/v3/role_assignments?user.id=8293dab3ffd647f5927d323572fdbdce&scope.project.id=daf12585428b41a2b45bdbc85413a6c4&role.id=759d845eb6284e42931b4265af448ebc => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:04:35.463469 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a647244-9f94-45c1-a9fa-49cc8f93899a None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:35.464270 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a647244-9f94-45c1-a9fa-49cc8f93899a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:35.464270 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a647244-9f94-45c1-a9fa-49cc8f93899a None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:35.465584 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 622/1261] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:04:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:04:35.470786 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b1347-a2a2-49e4-9dac-a19eac7b7e94 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:35.471206 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b1347-a2a2-49e4-9dac-a19eac7b7e94 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:35.471364 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b1347-a2a2-49e4-9dac-a19eac7b7e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:35.488291 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-ff6b1347-a2a2-49e4-9dac-a19eac7b7e94 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:04:35.489442 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ff6b1347-a2a2-49e4-9dac-a19eac7b7e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:04:35.553239 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 640/1262] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:04:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:04:35.575773 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a647244-9f94-45c1-a9fa-49cc8f93899a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:35.586448 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:35.676335 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a647244-9f94-45c1-a9fa-49cc8f93899a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:35.708381 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e200b887-e34b-4285-8e93-e28779056e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:35.709154 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e200b887-e34b-4285-8e93-e28779056e5e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:35.709436 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e200b887-e34b-4285-8e93-e28779056e5e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:35.709713 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e200b887-e34b-4285-8e93-e28779056e5e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:35.720525 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e200b887-e34b-4285-8e93-e28779056e5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:04:35.728661 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-e200b887-e34b-4285-8e93-e28779056e5e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:04:35.728661 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:04:35.728661 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:04:35.728661 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:04:35.728661 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:04:35.728661 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:04:35.728661 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:04:35.728661 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:04:35.728661 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:04:35.728661 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:04:35.728661 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:04:35.728661 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:04:35.728661 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:04:35.728661 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:04:35.728661 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:04:35.728661 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:04:35.729795 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:04:35.729795 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:04:35.729795 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:04:35.729795 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:04:35.729795 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:04:35.729795 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:04:35.729795 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:04:35.729795 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:04:35.729795 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:04:35.729795 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:04:35.729795 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:04:35.729795 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:04:35.729795 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:04:35.729795 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:04:35.729795 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:04:35.729795 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:04:35.731026 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:04:35.731026 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:04:35.731026 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:04:35.731026 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:04:35.731026 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:04:35.731026 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:04:35.731026 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:04:35.731026 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:04:35.731026 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:04:35.731026 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:04:35.731026 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:04:35.731026 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:04:35.731026 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 623/1263] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 15:04:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:04:35.735821 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6b1347-a2a2-49e4-9dac-a19eac7b7e94 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:35.753289 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:35.830375 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6b1347-a2a2-49e4-9dac-a19eac7b7e94 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:35.863277 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb0ea05-2bf9-4cee-bde8-f6f6d25dc435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:35.863929 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0ea05-2bf9-4cee-bde8-f6f6d25dc435 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:35.864212 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0ea05-2bf9-4cee-bde8-f6f6d25dc435 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:35.864488 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0ea05-2bf9-4cee-bde8-f6f6d25dc435 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:35.865262 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb0ea05-2bf9-4cee-bde8-f6f6d25dc435 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:04:35.865585 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb0ea05-2bf9-4cee-bde8-f6f6d25dc435 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:35.867083 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb0ea05-2bf9-4cee-bde8-f6f6d25dc435 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:35.876379 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 641/1264] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:04:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:04:35.882345 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e200b887-e34b-4285-8e93-e28779056e5e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:35.958235 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:35.992075 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e200b887-e34b-4285-8e93-e28779056e5e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:36.023780 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5732610-68d0-4a9f-9cfb-7fae2e896d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:36.024551 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5732610-68d0-4a9f-9cfb-7fae2e896d62 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:36.025627 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5732610-68d0-4a9f-9cfb-7fae2e896d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:36.025900 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5732610-68d0-4a9f-9cfb-7fae2e896d62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:36.036002 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5732610-68d0-4a9f-9cfb-7fae2e896d62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:36.037430 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5732610-68d0-4a9f-9cfb-7fae2e896d62 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:36.054197 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-d5732610-68d0-4a9f-9cfb-7fae2e896d62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:04:36.054197 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:04:36.054197 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:04:36.054197 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:04:36.054197 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:04:36.054197 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:04:36.054197 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:04:36.054197 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:04:36.054197 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:04:36.054197 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:04:36.054197 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:04:36.054197 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:04:36.054197 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:04:36.054197 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:04:36.054197 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:04:36.055319 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:04:36.055319 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:04:36.055319 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:04:36.055319 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:04:36.055319 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:04:36.055319 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:04:36.055319 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:04:36.055319 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:04:36.055319 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:04:36.055319 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:04:36.055319 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:04:36.055319 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:04:36.055319 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:04:36.055319 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:04:36.055319 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:04:36.055319 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:04:36.055319 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:04:36.056628 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:04:36.056628 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:04:36.056628 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:04:36.056628 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:04:36.056628 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 624/1265] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:04:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:04:36.061252 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb0ea05-2bf9-4cee-bde8-f6f6d25dc435 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:36.139612 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:36.159646 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb0ea05-2bf9-4cee-bde8-f6f6d25dc435 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:36.193335 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa75b69-ae43-4878-b112-39332b36e446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:36.193988 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa75b69-ae43-4878-b112-39332b36e446 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:36.194270 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa75b69-ae43-4878-b112-39332b36e446 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:36.194543 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa75b69-ae43-4878-b112-39332b36e446 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:36.195282 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa75b69-ae43-4878-b112-39332b36e446 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:04:36.195586 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa75b69-ae43-4878-b112-39332b36e446 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:36.197806 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa75b69-ae43-4878-b112-39332b36e446 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:36.209881 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 642/1266] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:04:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:04:36.215997 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5732610-68d0-4a9f-9cfb-7fae2e896d62 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:36.312732 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5732610-68d0-4a9f-9cfb-7fae2e896d62 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:36.332532 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:36.346484 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95b1de-4b4a-4d7e-9b2f-061476e3b355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:36.347295 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95b1de-4b4a-4d7e-9b2f-061476e3b355 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:36.347581 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95b1de-4b4a-4d7e-9b2f-061476e3b355 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:36.347853 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95b1de-4b4a-4d7e-9b2f-061476e3b355 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:36.355129 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f95b1de-4b4a-4d7e-9b2f-061476e3b355 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:36.356554 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f95b1de-4b4a-4d7e-9b2f-061476e3b355 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:36.370998 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-6f95b1de-4b4a-4d7e-9b2f-061476e3b355 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 15:04:36.370998 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:04:36.370998 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:04:36.370998 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:04:36.370998 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:04:36.370998 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:04:36.370998 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:04:36.370998 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:04:36.370998 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:04:36.370998 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:04:36.370998 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:04:36.370998 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:04:36.370998 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 15:04:36.370998 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 15:04:36.370998 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 15:04:36.370998 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 15:04:36.372111 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:04:36.372111 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:04:36.372111 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 15:04:36.372111 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:04:36.372111 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 15:04:36.372111 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:04:36.372111 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:04:36.372111 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:04:36.372111 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:04:36.372111 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:04:36.372111 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:04:36.372111 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:04:36.372111 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:04:36.372111 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:04:36.372111 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:04:36.372111 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:04:36.372111 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:04:36.372111 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:04:36.373279 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:04:36.373279 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:04:36.373279 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 15:04:36.373279 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 15:04:36.373279 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 15:04:36.373279 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 15:04:36.373279 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 15:04:36.373279 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 15:04:36.373279 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 15:04:36.373279 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:04:36.373279 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 625/1267] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:04:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:04:36.378012 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa75b69-ae43-4878-b112-39332b36e446 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:36.484393 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa75b69-ae43-4878-b112-39332b36e446 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:36.518048 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac713e-944a-47f7-b350-6136a3dc63bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:36.518711 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac713e-944a-47f7-b350-6136a3dc63bf None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:36.519029 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac713e-944a-47f7-b350-6136a3dc63bf None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:36.519309 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac713e-944a-47f7-b350-6136a3dc63bf None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:36.520139 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fac713e-944a-47f7-b350-6136a3dc63bf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:04:36.520479 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fac713e-944a-47f7-b350-6136a3dc63bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:36.521891 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fac713e-944a-47f7-b350-6136a3dc63bf None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:36.530347 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:36.538609 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 643/1268] 10.222.0.50 () {56 vars in 1143 bytes} [Fri May 27 15:04:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:04:36.544851 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95b1de-4b4a-4d7e-9b2f-061476e3b355 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:36.641694 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95b1de-4b4a-4d7e-9b2f-061476e3b355 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:36.673845 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672a567b-23c2-4ee7-8258-5480ff35bf8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:36.674613 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a567b-23c2-4ee7-8258-5480ff35bf8c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:36.674922 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a567b-23c2-4ee7-8258-5480ff35bf8c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:36.675204 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a567b-23c2-4ee7-8258-5480ff35bf8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:36.686021 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672a567b-23c2-4ee7-8258-5480ff35bf8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:36.687490 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672a567b-23c2-4ee7-8258-5480ff35bf8c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:36.704271 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-672a567b-23c2-4ee7-8258-5480ff35bf8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:04:36.704271 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:04:36.704271 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:04:36.704271 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:04:36.704271 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:04:36.704271 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:04:36.704271 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:04:36.704271 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:04:36.704271 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:04:36.704271 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:04:36.704271 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:04:36.704271 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:04:36.704271 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:04:36.704271 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:04:36.704271 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:04:36.705316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:04:36.705316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:04:36.705316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:04:36.705316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:04:36.705316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:04:36.705316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:04:36.705316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:04:36.705316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:04:36.705316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:04:36.705316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:04:36.705316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:04:36.705316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:04:36.705316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:04:36.705316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:04:36.705316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:04:36.705316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:04:36.705316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:04:36.706438 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:04:36.706438 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:04:36.706438 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:04:36.706438 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:04:36.706438 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 626/1269] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:04:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:04:36.711179 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac713e-944a-47f7-b350-6136a3dc63bf None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:36.808299 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac713e-944a-47f7-b350-6136a3dc63bf None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:36.841152 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c253d9a0-79b9-487e-9879-54f5486465d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:36.841803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c253d9a0-79b9-487e-9879-54f5486465d3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:36.842088 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c253d9a0-79b9-487e-9879-54f5486465d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:36.842365 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c253d9a0-79b9-487e-9879-54f5486465d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:36.843108 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c253d9a0-79b9-487e-9879-54f5486465d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:04:36.843421 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c253d9a0-79b9-487e-9879-54f5486465d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:36.844773 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c253d9a0-79b9-487e-9879-54f5486465d3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:36.856285 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 644/1270] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:04:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:04:36.861892 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672a567b-23c2-4ee7-8258-5480ff35bf8c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:36.895382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:36.961384 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672a567b-23c2-4ee7-8258-5480ff35bf8c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:36.994485 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17405437-0bf6-448b-b6a3-5665c3e53097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:36.995279 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17405437-0bf6-448b-b6a3-5665c3e53097 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:36.995561 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17405437-0bf6-448b-b6a3-5665c3e53097 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:36.995832 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17405437-0bf6-448b-b6a3-5665c3e53097 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:37.002162 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17405437-0bf6-448b-b6a3-5665c3e53097 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:37.003608 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17405437-0bf6-448b-b6a3-5665c3e53097 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:37.017284 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-17405437-0bf6-448b-b6a3-5665c3e53097 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:04:37.017284 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:04:37.017284 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:04:37.017284 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:04:37.017284 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:04:37.017284 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:04:37.017284 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:04:37.017284 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:04:37.017284 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:04:37.017284 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:04:37.017284 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:04:37.017284 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:04:37.017284 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 15:04:37.017284 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 15:04:37.017284 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 15:04:37.018314 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 15:04:37.018314 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:04:37.018314 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:04:37.018314 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:04:37.018314 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:04:37.018314 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:04:37.018314 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:04:37.018314 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:04:37.018314 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:04:37.018314 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:04:37.018314 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:04:37.018314 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:04:37.018314 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:04:37.018314 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:04:37.018314 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:04:37.018314 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:04:37.018314 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:04:37.019405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 15:04:37.019405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 15:04:37.019405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 15:04:37.019405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 15:04:37.019405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 15:04:37.019405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 15:04:37.019405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:04:37.019405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:04:37.019405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 627/1271] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:04:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:04:37.024214 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c253d9a0-79b9-487e-9879-54f5486465d3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:37.081026 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:37.122449 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c253d9a0-79b9-487e-9879-54f5486465d3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:37.155115 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e943a4-4b92-4d9f-818e-3a6ed5212744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:37.155774 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e943a4-4b92-4d9f-818e-3a6ed5212744 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:37.156060 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e943a4-4b92-4d9f-818e-3a6ed5212744 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:37.156337 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e943a4-4b92-4d9f-818e-3a6ed5212744 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:37.157121 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e943a4-4b92-4d9f-818e-3a6ed5212744 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:04:37.157427 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e943a4-4b92-4d9f-818e-3a6ed5212744 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:37.158874 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e943a4-4b92-4d9f-818e-3a6ed5212744 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:37.172215 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 645/1272] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 15:04:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:04:37.178056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17405437-0bf6-448b-b6a3-5665c3e53097 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:37.268275 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:37.277568 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17405437-0bf6-448b-b6a3-5665c3e53097 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:37.310265 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b8111a-6a73-4d79-81d9-62894590e3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:37.311160 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8111a-6a73-4d79-81d9-62894590e3ef None admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:37.311439 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8111a-6a73-4d79-81d9-62894590e3ef None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:37.311709 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8111a-6a73-4d79-81d9-62894590e3ef None admin] PATH_INFO: `/v3/projects/daf12585428b41a2b45bdbc85413a6c4/users/8293dab3ffd647f5927d323572fdbdce/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:37.313606 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b8111a-6a73-4d79-81d9-62894590e3ef None admin] RBAC: Authorizing `identity:create_grant(project_id=daf12585428b41a2b45bdbc85413a6c4, user_id=8293dab3ffd647f5927d323572fdbdce, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:37.315051 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b8111a-6a73-4d79-81d9-62894590e3ef None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:37.327817 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 628/1273] 10.222.0.50 () {58 vars in 1439 bytes} [Fri May 27 15:04:37 2022] PUT /identity/v3/projects/daf12585428b41a2b45bdbc85413a6c4/users/8293dab3ffd647f5927d323572fdbdce/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:04:39.499995 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75ae60-fe5b-4f30-902d-eac5b10042b7 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:39.500542 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75ae60-fe5b-4f30-902d-eac5b10042b7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:39.500542 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d75ae60-fe5b-4f30-902d-eac5b10042b7 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:39.501814 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 646/1274] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:04:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:04:39.506936 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f010e60-ba64-45e5-8a28-cd344392aac7 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:39.507389 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f010e60-ba64-45e5-8a28-cd344392aac7 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:39.507494 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f010e60-ba64-45e5-8a28-cd344392aac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:39.523977 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2f010e60-ba64-45e5-8a28-cd344392aac7 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:04:39.525741 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2f010e60-ba64-45e5-8a28-cd344392aac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:04:39.591332 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 629/1275] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:04:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:04:39.614034 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d75ae60-fe5b-4f30-902d-eac5b10042b7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:39.709862 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d75ae60-fe5b-4f30-902d-eac5b10042b7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:39.726931 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:39.742616 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707fe1f-ec0e-4757-b4ce-31818b22261a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:39.743412 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707fe1f-ec0e-4757-b4ce-31818b22261a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:39.743728 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707fe1f-ec0e-4757-b4ce-31818b22261a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:39.743996 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707fe1f-ec0e-4757-b4ce-31818b22261a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:39.753628 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4707fe1f-ec0e-4757-b4ce-31818b22261a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 27 15:04:39.761766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-4707fe1f-ec0e-4757-b4ce-31818b22261a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:04:39.761766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:04:39.761766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:04:39.761766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:04:39.761766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:04:39.761766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:04:39.761766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:04:39.761766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:04:39.761766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:04:39.761766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:04:39.761766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:04:39.761766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:04:39.761766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 27 15:04:39.761766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 27 15:04:39.761766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 27 15:04:39.761766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 27 15:04:39.762974 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 27 15:04:39.762974 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 27 15:04:39.762974 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 27 15:04:39.762974 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 27 15:04:39.762974 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:04:39.762974 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:04:39.762974 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:04:39.762974 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:04:39.762974 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:04:39.762974 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:04:39.762974 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:04:39.762974 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:04:39.762974 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:04:39.762974 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:04:39.762974 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:04:39.762974 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:04:39.764222 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:04:39.764222 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:04:39.764222 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:04:39.764222 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:04:39.764222 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 27 15:04:39.764222 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 27 15:04:39.764222 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 27 15:04:39.764222 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 27 15:04:39.764222 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 27 15:04:39.764222 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 27 15:04:39.764222 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 27 15:04:39.764222 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:04:39.764222 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 647/1276] 10.222.0.50 () {56 vars in 1096 bytes} [Fri May 27 15:04:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:04:39.768669 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f010e60-ba64-45e5-8a28-cd344392aac7 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:39.846255 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:39.864525 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f010e60-ba64-45e5-8a28-cd344392aac7 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:39.907174 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fece20b8-f3d0-41b4-a1dc-19344b7ff8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:39.908026 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fece20b8-f3d0-41b4-a1dc-19344b7ff8e6 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:39.908377 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fece20b8-f3d0-41b4-a1dc-19344b7ff8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:39.908741 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fece20b8-f3d0-41b4-a1dc-19344b7ff8e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:39.909668 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fece20b8-f3d0-41b4-a1dc-19344b7ff8e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:04:39.910027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fece20b8-f3d0-41b4-a1dc-19344b7ff8e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:39.911806 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fece20b8-f3d0-41b4-a1dc-19344b7ff8e6 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:39.922229 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 630/1277] 10.222.0.50 () {56 vars in 1099 bytes} [Fri May 27 15:04:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:04:39.928072 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707fe1f-ec0e-4757-b4ce-31818b22261a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:40.030913 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707fe1f-ec0e-4757-b4ce-31818b22261a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:40.063007 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dced22-9442-4cd7-8eef-edd7067fa70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:40.063786 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dced22-9442-4cd7-8eef-edd7067fa70d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:40.064073 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dced22-9442-4cd7-8eef-edd7067fa70d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:40.064351 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dced22-9442-4cd7-8eef-edd7067fa70d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:40.075257 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dced22-9442-4cd7-8eef-edd7067fa70d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:40.076678 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dced22-9442-4cd7-8eef-edd7067fa70d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:40.093590 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-56dced22-9442-4cd7-8eef-edd7067fa70d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:04:40.093590 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:04:40.093590 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:04:40.093590 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:04:40.093590 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:04:40.093590 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:04:40.093590 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:04:40.093590 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:04:40.093590 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:04:40.093590 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:04:40.093590 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:04:40.093590 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:04:40.093590 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:04:40.093590 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:04:40.093590 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:04:40.094649 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:04:40.094649 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:04:40.094649 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:04:40.094649 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:04:40.094649 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:04:40.094649 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:04:40.094649 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:04:40.094649 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:04:40.094649 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:04:40.094649 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:04:40.094649 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:04:40.094649 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:04:40.094649 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:04:40.094649 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:04:40.094649 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:04:40.094649 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:04:40.094649 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:04:40.095820 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:04:40.095820 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:04:40.095820 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:04:40.095820 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:04:40.095820 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 648/1278] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:04:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:04:40.100543 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fece20b8-f3d0-41b4-a1dc-19344b7ff8e6 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:40.196553 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fece20b8-f3d0-41b4-a1dc-19344b7ff8e6 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:40.228887 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e3b32c-cc6c-42f6-9e00-d701359fa224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:40.229532 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e3b32c-cc6c-42f6-9e00-d701359fa224 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:40.229817 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e3b32c-cc6c-42f6-9e00-d701359fa224 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:40.230092 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e3b32c-cc6c-42f6-9e00-d701359fa224 None admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:40.230870 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e3b32c-cc6c-42f6-9e00-d701359fa224 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:04:40.231180 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e3b32c-cc6c-42f6-9e00-d701359fa224 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:40.232539 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e3b32c-cc6c-42f6-9e00-d701359fa224 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:40.244504 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 631/1279] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:04:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:04:40.250224 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dced22-9442-4cd7-8eef-edd7067fa70d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:40.280548 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:40.349220 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dced22-9442-4cd7-8eef-edd7067fa70d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:40.382200 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8abb038-32f8-4535-a582-98dff081ee7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:40.383014 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8abb038-32f8-4535-a582-98dff081ee7d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:40.383300 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8abb038-32f8-4535-a582-98dff081ee7d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:40.383577 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8abb038-32f8-4535-a582-98dff081ee7d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:40.391180 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8abb038-32f8-4535-a582-98dff081ee7d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:40.392603 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8abb038-32f8-4535-a582-98dff081ee7d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:40.406679 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-b8abb038-32f8-4535-a582-98dff081ee7d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 27 15:04:40.406679 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:04:40.406679 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:04:40.406679 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:04:40.406679 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:04:40.406679 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:04:40.406679 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:04:40.406679 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:04:40.406679 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:04:40.406679 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:04:40.406679 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:04:40.406679 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:04:40.406679 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 27 15:04:40.406679 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 27 15:04:40.406679 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 27 15:04:40.406679 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 27 15:04:40.407907 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:04:40.407907 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:04:40.407907 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 27 15:04:40.407907 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:04:40.407907 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 27 15:04:40.407907 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 27 15:04:40.407907 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:04:40.407907 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:04:40.407907 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:04:40.407907 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:04:40.407907 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:04:40.407907 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:04:40.407907 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:04:40.407907 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:04:40.407907 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:04:40.407907 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:04:40.407907 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:04:40.407907 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:04:40.409254 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:04:40.409254 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:04:40.409254 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 27 15:04:40.409254 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 27 15:04:40.409254 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 27 15:04:40.409254 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 27 15:04:40.409254 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 27 15:04:40.409254 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 27 15:04:40.409254 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 27 15:04:40.409254 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:04:40.409254 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 649/1280] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:04:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:04:40.413750 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e3b32c-cc6c-42f6-9e00-d701359fa224 None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:40.418250 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:40.519438 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e3b32c-cc6c-42f6-9e00-d701359fa224 None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:40.551643 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b05071-ffc9-44ee-b9a2-545e7194f79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:40.552296 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b05071-ffc9-44ee-b9a2-545e7194f79c None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:40.552575 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b05071-ffc9-44ee-b9a2-545e7194f79c None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:40.552843 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b05071-ffc9-44ee-b9a2-545e7194f79c None admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:40.553656 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b05071-ffc9-44ee-b9a2-545e7194f79c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:04:40.553957 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b05071-ffc9-44ee-b9a2-545e7194f79c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:40.555392 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b05071-ffc9-44ee-b9a2-545e7194f79c None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:40.572744 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 632/1281] 10.222.0.50 () {56 vars in 1143 bytes} [Fri May 27 15:04:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:04:40.578459 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8abb038-32f8-4535-a582-98dff081ee7d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:40.667142 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:40.676998 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8abb038-32f8-4535-a582-98dff081ee7d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:40.710094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2789c1c9-e71a-44d1-bfae-384e6c744aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:40.710747 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2789c1c9-e71a-44d1-bfae-384e6c744aca None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:40.711051 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2789c1c9-e71a-44d1-bfae-384e6c744aca None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:40.711325 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2789c1c9-e71a-44d1-bfae-384e6c744aca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:40.788923 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2789c1c9-e71a-44d1-bfae-384e6c744aca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:40.791022 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2789c1c9-e71a-44d1-bfae-384e6c744aca None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:40.879817 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:40.890189 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 650/1282] 10.222.0.50 () {58 vars in 1282 bytes} [Fri May 27 15:04:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 27 15:04:40.896027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b05071-ffc9-44ee-b9a2-545e7194f79c None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:40.954752 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:40.993173 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b05071-ffc9-44ee-b9a2-545e7194f79c None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:41.026324 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca31a8-6165-43e1-b819-33e98621e01e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:41.027126 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca31a8-6165-43e1-b819-33e98621e01e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:41.027413 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca31a8-6165-43e1-b819-33e98621e01e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:41.027687 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca31a8-6165-43e1-b819-33e98621e01e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:41.037371 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bca31a8-6165-43e1-b819-33e98621e01e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:41.038819 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bca31a8-6165-43e1-b819-33e98621e01e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:41.056563 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-6bca31a8-6165-43e1-b819-33e98621e01e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:04:41.056563 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:04:41.056563 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:04:41.056563 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:04:41.056563 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:04:41.056563 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:04:41.056563 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:04:41.056563 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:04:41.056563 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:04:41.056563 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:04:41.056563 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:04:41.056563 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:04:41.056563 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 27 15:04:41.056563 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 27 15:04:41.056563 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 27 15:04:41.057650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 27 15:04:41.057650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:04:41.057650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:04:41.057650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:04:41.057650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:04:41.057650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:04:41.057650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:04:41.057650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:04:41.057650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:04:41.057650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:04:41.057650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:04:41.057650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:04:41.057650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:04:41.057650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:04:41.057650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:04:41.057650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:04:41.057650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:04:41.058985 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 27 15:04:41.058985 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 27 15:04:41.058985 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 27 15:04:41.058985 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:04:41.058985 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 633/1283] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:04:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:04:41.063286 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2789c1c9-e71a-44d1-bfae-384e6c744aca None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:41.167917 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2789c1c9-e71a-44d1-bfae-384e6c744aca None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:41.200971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f24db9f-1c35-487c-9943-364cd99de280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:41.201628 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f24db9f-1c35-487c-9943-364cd99de280 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:41.201912 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f24db9f-1c35-487c-9943-364cd99de280 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:41.202189 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f24db9f-1c35-487c-9943-364cd99de280 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:41.202927 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f24db9f-1c35-487c-9943-364cd99de280 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:04:41.203238 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f24db9f-1c35-487c-9943-364cd99de280 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:41.204601 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f24db9f-1c35-487c-9943-364cd99de280 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:41.216401 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 651/1284] 10.222.0.50 () {56 vars in 1109 bytes} [Fri May 27 15:04:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:04:41.221842 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca31a8-6165-43e1-b819-33e98621e01e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:41.320882 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca31a8-6165-43e1-b819-33e98621e01e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:41.334659 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:41.355113 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b690ae-e4c0-4843-9f8b-e7270f3f831b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:41.355889 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b690ae-e4c0-4843-9f8b-e7270f3f831b None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:41.356173 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b690ae-e4c0-4843-9f8b-e7270f3f831b None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:41.356448 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b690ae-e4c0-4843-9f8b-e7270f3f831b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:41.362783 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b690ae-e4c0-4843-9f8b-e7270f3f831b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:41.364238 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b690ae-e4c0-4843-9f8b-e7270f3f831b None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:41.377565 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application [None req-c8b690ae-e4c0-4843-9f8b-e7270f3f831b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:04:41.377565 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:04:41.377565 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:04:41.377565 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:04:41.377565 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:04:41.377565 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:04:41.377565 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:04:41.377565 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:04:41.377565 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:04:41.377565 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:04:41.377565 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:04:41.377565 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:04:41.377565 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 27 15:04:41.377565 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(project_id) May 27 15:04:41.377565 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 27 15:04:41.378719 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 27 15:04:41.378719 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:04:41.378719 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:04:41.378719 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:04:41.378719 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:04:41.378719 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:04:41.378719 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:04:41.378719 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:04:41.378719 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application with Lock( May 27 15:04:41.378719 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:04:41.378719 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._enter() May 27 15:04:41.378719 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:04:41.378719 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:04:41.378719 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:04:41.378719 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.creator() May 27 15:04:41.378719 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:04:41.378719 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application created_value = creator( May 27 15:04:41.379729 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 27 15:04:41.379729 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 27 15:04:41.379729 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 27 15:04:41.379729 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 27 15:04:41.379729 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 27 15:04:41.379729 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 27 15:04:41.379729 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 27 15:04:41.379729 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: ERROR keystone.server.flask.application May 27 15:04:41.379729 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 634/1285] 10.222.0.50 () {56 vars in 1111 bytes} [Fri May 27 15:04:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:04:41.385192 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f24db9f-1c35-487c-9943-364cd99de280 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:41.433677 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:41.483585 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f24db9f-1c35-487c-9943-364cd99de280 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:41.516137 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56ab4ac-f817-4368-b605-91d96b09bd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:41.516784 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ab4ac-f817-4368-b605-91d96b09bd63 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:41.517065 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ab4ac-f817-4368-b605-91d96b09bd63 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:41.517339 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56ab4ac-f817-4368-b605-91d96b09bd63 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:41.518110 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56ab4ac-f817-4368-b605-91d96b09bd63 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:04:41.518415 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56ab4ac-f817-4368-b605-91d96b09bd63 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:41.519850 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56ab4ac-f817-4368-b605-91d96b09bd63 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:41.531426 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 652/1286] 10.222.0.50 () {56 vars in 1148 bytes} [Fri May 27 15:04:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:04:41.537605 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b690ae-e4c0-4843-9f8b-e7270f3f831b None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:41.634513 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b690ae-e4c0-4843-9f8b-e7270f3f831b None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:41.667119 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce953f7-2413-40f7-aac4-84931ed55877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:41.667753 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce953f7-2413-40f7-aac4-84931ed55877 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:41.668042 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce953f7-2413-40f7-aac4-84931ed55877 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:41.668315 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce953f7-2413-40f7-aac4-84931ed55877 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:41.669122 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce953f7-2413-40f7-aac4-84931ed55877 None admin] RBAC: Adding query filter params (role.id=759d845eb6284e42931b4265af448ebc, scope.project.id=daf12585428b41a2b45bdbc85413a6c4, user.id=8293dab3ffd647f5927d323572fdbdce) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:04:41.669432 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce953f7-2413-40f7-aac4-84931ed55877 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:41.670871 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce953f7-2413-40f7-aac4-84931ed55877 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:41.676502 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 635/1287] 10.222.0.50 () {56 vars in 1374 bytes} [Fri May 27 15:04:41 2022] GET /identity/v3/role_assignments?user.id=8293dab3ffd647f5927d323572fdbdce&scope.project.id=daf12585428b41a2b45bdbc85413a6c4&role.id=759d845eb6284e42931b4265af448ebc => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:04:43.893434 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee2b6f-47c4-4d5e-b4c4-a29e3db22152 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:43.893987 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee2b6f-47c4-4d5e-b4c4-a29e3db22152 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:43.893987 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee2b6f-47c4-4d5e-b4c4-a29e3db22152 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:43.895709 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 653/1288] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:04:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 15:04:43.902455 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8a5f1-846b-45aa-82fe-ae9efe6f1e65 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:43.903049 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8a5f1-846b-45aa-82fe-ae9efe6f1e65 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:43.903049 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8a5f1-846b-45aa-82fe-ae9efe6f1e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:43.920904 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-33b8a5f1-846b-45aa-82fe-ae9efe6f1e65 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:04:43.922098 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-33b8a5f1-846b-45aa-82fe-ae9efe6f1e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:04:43.952143 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:43.989941 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 636/1289] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:04:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 15:04:44.011836 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee2b6f-47c4-4d5e-b4c4-a29e3db22152 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:44.110586 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee2b6f-47c4-4d5e-b4c4-a29e3db22152 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:44.121120 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:44.144159 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f8f3c-0506-45e9-be0c-b3ba2f06dbcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:44.144881 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f8f3c-0506-45e9-be0c-b3ba2f06dbcf None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:44.145167 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f8f3c-0506-45e9-be0c-b3ba2f06dbcf None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:44.145446 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f8f3c-0506-45e9-be0c-b3ba2f06dbcf None admin] PATH_INFO: `/v3/services/placement` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:44.146086 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80f8f3c-0506-45e9-be0c-b3ba2f06dbcf None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:44.147494 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80f8f3c-0506-45e9-be0c-b3ba2f06dbcf None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:44.160068 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-d80f8f3c-0506-45e9-be0c-b3ba2f06dbcf None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 27 15:04:44.160068 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:04:44.160068 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 15:04:44.160068 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 15:04:44.160068 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 15:04:44.160068 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 15:04:44.160068 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 15:04:44.160068 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:04:44.160068 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 15:04:44.160068 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:04:44.160068 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 15:04:44.160068 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:04:44.160068 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:04:44.160068 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:04:44.160068 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:04:44.160068 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:04:44.160068 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:04:44.161329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:04:44.161329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:04:44.161329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:04:44.161329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:04:44.161329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:04:44.161329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:04:44.161329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 15:04:44.161329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 15:04:44.161329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 15:04:44.161329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 15:04:44.161329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:04:44.161329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:04:44.161329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:04:44.161329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:04:44.161329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:04:44.161329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:04:44.162473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:04:44.162473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:04:44.162473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:04:44.162473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:04:44.162473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:04:44.162473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:04:44.162473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:04:44.162473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:04:44.162473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:04:44.162473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:04:44.162473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 15:04:44.162473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:04:44.162473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 15:04:44.162473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:04:44.162473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 654/1290] 10.222.0.50 () {56 vars in 1117 bytes} [Fri May 27 15:04:44 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:04:44.167528 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8a5f1-846b-45aa-82fe-ae9efe6f1e65 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:44.261410 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8a5f1-846b-45aa-82fe-ae9efe6f1e65 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:44.294297 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd6ca4b-bdf9-4a8f-bf9c-7e973a7e60cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:44.295010 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd6ca4b-bdf9-4a8f-bf9c-7e973a7e60cf None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:44.295296 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd6ca4b-bdf9-4a8f-bf9c-7e973a7e60cf None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:44.295572 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd6ca4b-bdf9-4a8f-bf9c-7e973a7e60cf None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:44.296270 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd6ca4b-bdf9-4a8f-bf9c-7e973a7e60cf None admin] RBAC: Adding query filter params (name=placement) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:04:44.296577 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd6ca4b-bdf9-4a8f-bf9c-7e973a7e60cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:44.297930 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd6ca4b-bdf9-4a8f-bf9c-7e973a7e60cf None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:44.302610 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 637/1291] 10.222.0.50 () {56 vars in 1116 bytes} [Fri May 27 15:04:44 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:04:44.308358 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f8f3c-0506-45e9-be0c-b3ba2f06dbcf None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:44.405693 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f8f3c-0506-45e9-be0c-b3ba2f06dbcf None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:44.441420 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faf5f86-4cb8-4140-94db-9c89dc5066e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:44.442077 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf5f86-4cb8-4140-94db-9c89dc5066e1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:44.442361 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf5f86-4cb8-4140-94db-9c89dc5066e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:44.442661 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf5f86-4cb8-4140-94db-9c89dc5066e1 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:44.443393 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faf5f86-4cb8-4140-94db-9c89dc5066e1 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:04:44.443700 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faf5f86-4cb8-4140-94db-9c89dc5066e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:44.445052 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faf5f86-4cb8-4140-94db-9c89dc5066e1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:44.450154 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 655/1292] 10.222.0.50 () {56 vars in 1116 bytes} [Fri May 27 15:04:44 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:04:46.644543 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02634d-e373-42da-bec2-0844258fdaf2 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:46.645101 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02634d-e373-42da-bec2-0844258fdaf2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:46.645101 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02634d-e373-42da-bec2-0844258fdaf2 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:46.646456 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 638/1293] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:04:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:04:46.652441 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9434c855-b07d-4c67-bd3b-ce45bd810bc2 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:46.652850 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9434c855-b07d-4c67-bd3b-ce45bd810bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:46.653003 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9434c855-b07d-4c67-bd3b-ce45bd810bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:46.670090 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9434c855-b07d-4c67-bd3b-ce45bd810bc2 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:04:46.671319 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9434c855-b07d-4c67-bd3b-ce45bd810bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:04:46.701201 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:46.737831 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 656/1294] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:04:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 27 15:04:46.761291 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba02634d-e373-42da-bec2-0844258fdaf2 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:46.792285 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:46.862603 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba02634d-e373-42da-bec2-0844258fdaf2 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:46.903909 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5040730d-e9dd-4bf8-a692-d7a9c000142a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:46.904681 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5040730d-e9dd-4bf8-a692-d7a9c000142a None admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:46.905055 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5040730d-e9dd-4bf8-a692-d7a9c000142a None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:46.905390 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5040730d-e9dd-4bf8-a692-d7a9c000142a None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:46.906359 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5040730d-e9dd-4bf8-a692-d7a9c000142a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:46.908161 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5040730d-e9dd-4bf8-a692-d7a9c000142a None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:46.923843 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 639/1295] 10.222.0.50 () {60 vars in 1141 bytes} [Fri May 27 15:04:46 2022] POST /identity/v3/services => generated 246 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:04:49.126530 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767d9d8-52b5-4637-9883-041406ce914d None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:49.127100 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767d9d8-52b5-4637-9883-041406ce914d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:49.127100 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1767d9d8-52b5-4637-9883-041406ce914d None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:49.128476 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 657/1296] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:04:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:04:49.133765 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c10584-13eb-493b-bd36-13ea4e349cf5 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:49.134212 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c10584-13eb-493b-bd36-13ea4e349cf5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:49.134502 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c10584-13eb-493b-bd36-13ea4e349cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:49.151706 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-63c10584-13eb-493b-bd36-13ea4e349cf5 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:04:49.152869 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-63c10584-13eb-493b-bd36-13ea4e349cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:04:49.224723 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 640/1297] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:04:49 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:04:49.247871 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1767d9d8-52b5-4637-9883-041406ce914d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:49.349062 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1767d9d8-52b5-4637-9883-041406ce914d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:49.381719 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f9163-b3b5-4fca-a770-8018fe32b367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:49.382446 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f9163-b3b5-4fca-a770-8018fe32b367 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:49.382731 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f9163-b3b5-4fca-a770-8018fe32b367 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:49.383042 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f9163-b3b5-4fca-a770-8018fe32b367 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:49.383693 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172f9163-b3b5-4fca-a770-8018fe32b367 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:49.385738 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172f9163-b3b5-4fca-a770-8018fe32b367 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:49.397340 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-172f9163-b3b5-4fca-a770-8018fe32b367 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 27 15:04:49.397340 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:04:49.397340 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 15:04:49.397340 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 15:04:49.397340 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 15:04:49.397340 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 15:04:49.397340 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 15:04:49.397340 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:04:49.397340 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 15:04:49.397340 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:04:49.397340 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 15:04:49.397340 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:04:49.397340 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:04:49.397340 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:04:49.397340 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:04:49.397340 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:04:49.397340 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:04:49.398398 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:04:49.398398 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:04:49.398398 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:04:49.398398 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:04:49.398398 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:04:49.398398 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:04:49.398398 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 15:04:49.398398 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 15:04:49.398398 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 15:04:49.398398 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 15:04:49.398398 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:04:49.398398 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:04:49.398398 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:04:49.398398 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:04:49.398398 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:04:49.398398 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:04:49.399417 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:04:49.399417 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:04:49.399417 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:04:49.399417 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:04:49.399417 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:04:49.399417 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:04:49.399417 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:04:49.399417 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:04:49.399417 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:04:49.399417 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:04:49.399417 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 15:04:49.399417 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:04:49.399417 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 15:04:49.399417 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:04:49.399417 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 658/1298] 10.222.0.50 () {56 vars in 1117 bytes} [Fri May 27 15:04:49 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:04:49.405260 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c10584-13eb-493b-bd36-13ea4e349cf5 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:49.422745 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:49.500947 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c10584-13eb-493b-bd36-13ea4e349cf5 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:49.533742 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173dd434-2a0b-4af8-8314-e5e47bbf629e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:49.534388 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173dd434-2a0b-4af8-8314-e5e47bbf629e None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:49.534666 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173dd434-2a0b-4af8-8314-e5e47bbf629e None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:49.534964 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173dd434-2a0b-4af8-8314-e5e47bbf629e None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:49.535656 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173dd434-2a0b-4af8-8314-e5e47bbf629e None admin] RBAC: Adding query filter params (name=placement) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:04:49.535957 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173dd434-2a0b-4af8-8314-e5e47bbf629e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:49.537314 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173dd434-2a0b-4af8-8314-e5e47bbf629e None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:49.542279 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 641/1299] 10.222.0.50 () {56 vars in 1116 bytes} [Fri May 27 15:04:49 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:04:49.548381 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f9163-b3b5-4fca-a770-8018fe32b367 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:49.579031 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:49.645549 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f9163-b3b5-4fca-a770-8018fe32b367 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:49.678170 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48dff5a-4ecb-443d-8f8c-8e4a0e713c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:49.678822 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48dff5a-4ecb-443d-8f8c-8e4a0e713c2b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:49.679138 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48dff5a-4ecb-443d-8f8c-8e4a0e713c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:49.679416 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48dff5a-4ecb-443d-8f8c-8e4a0e713c2b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:49.680199 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48dff5a-4ecb-443d-8f8c-8e4a0e713c2b None admin] RBAC: Adding query filter params (interface=public, service_id=3fc577dd3ba04cb3bcb8b00458a3f8e9, region_id=RegionOne) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:04:49.680513 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48dff5a-4ecb-443d-8f8c-8e4a0e713c2b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:49.682409 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48dff5a-4ecb-443d-8f8c-8e4a0e713c2b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:49.687351 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 659/1300] 10.222.0.50 () {56 vars in 1251 bytes} [Fri May 27 15:04:49 2022] GET /identity/v3/endpoints?service_id=3fc577dd3ba04cb3bcb8b00458a3f8e9&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:04:49.692985 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173dd434-2a0b-4af8-8314-e5e47bbf629e None admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:49.771108 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:49.788504 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173dd434-2a0b-4af8-8314-e5e47bbf629e None admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:49.820695 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90d376e-7291-4e94-b335-c8619ff72025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:49.821958 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d376e-7291-4e94-b335-c8619ff72025 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:49.822340 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d376e-7291-4e94-b335-c8619ff72025 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:49.822619 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d376e-7291-4e94-b335-c8619ff72025 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:49.823317 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90d376e-7291-4e94-b335-c8619ff72025 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:49.824662 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90d376e-7291-4e94-b335-c8619ff72025 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:49.831199 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 642/1301] 10.222.0.50 () {56 vars in 1087 bytes} [Fri May 27 15:04:49 2022] GET /identity/v3/services => generated 2191 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:04:52.041776 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1788431-764e-42ef-8576-92dd0043f25d None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:52.042343 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1788431-764e-42ef-8576-92dd0043f25d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:52.042343 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1788431-764e-42ef-8576-92dd0043f25d None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:52.043833 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 660/1302] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:04:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:04:52.049121 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db5aa0-ae76-4a30-9d84-f4c3a80d63a3 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:52.049486 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db5aa0-ae76-4a30-9d84-f4c3a80d63a3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:52.049674 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db5aa0-ae76-4a30-9d84-f4c3a80d63a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:52.066079 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c7db5aa0-ae76-4a30-9d84-f4c3a80d63a3 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:04:52.067278 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c7db5aa0-ae76-4a30-9d84-f4c3a80d63a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:04:52.131702 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 643/1303] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:04:52 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 27 15:04:52.159248 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1788431-764e-42ef-8576-92dd0043f25d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:52.250783 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:52.257570 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1788431-764e-42ef-8576-92dd0043f25d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:52.291355 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb90c93d-461f-44cc-be89-c7f7820580ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:52.292178 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb90c93d-461f-44cc-be89-c7f7820580ba None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:52.292438 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb90c93d-461f-44cc-be89-c7f7820580ba None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:52.292711 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb90c93d-461f-44cc-be89-c7f7820580ba None admin] PATH_INFO: `/v3/services/placement` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:52.293352 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb90c93d-461f-44cc-be89-c7f7820580ba None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:52.294707 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb90c93d-461f-44cc-be89-c7f7820580ba None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:52.306510 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-eb90c93d-461f-44cc-be89-c7f7820580ba None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 27 15:04:52.306510 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:04:52.306510 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 27 15:04:52.306510 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 27 15:04:52.306510 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 27 15:04:52.306510 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 27 15:04:52.306510 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 27 15:04:52.306510 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:04:52.306510 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 15:04:52.306510 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:04:52.306510 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 27 15:04:52.306510 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:04:52.306510 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 27 15:04:52.306510 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 27 15:04:52.306510 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 27 15:04:52.306510 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 27 15:04:52.306510 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 27 15:04:52.307724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 27 15:04:52.307724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 27 15:04:52.307724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 27 15:04:52.307724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 27 15:04:52.307724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 27 15:04:52.307724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 27 15:04:52.307724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 27 15:04:52.307724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) May 27 15:04:52.307724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 27 15:04:52.307724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 27 15:04:52.307724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 27 15:04:52.307724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 27 15:04:52.307724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 27 15:04:52.307724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 27 15:04:52.307724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 27 15:04:52.307724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 27 15:04:52.308777 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 27 15:04:52.308777 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 27 15:04:52.308777 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 27 15:04:52.308777 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 27 15:04:52.308777 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 27 15:04:52.308777 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 27 15:04:52.308777 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 27 15:04:52.308777 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 27 15:04:52.308777 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 27 15:04:52.308777 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 27 15:04:52.308777 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 27 15:04:52.308777 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 27 15:04:52.308777 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 27 15:04:52.308777 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 27 15:04:52.308777 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 661/1304] 10.222.0.50 () {56 vars in 1117 bytes} [Fri May 27 15:04:52 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 27 15:04:52.313875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7db5aa0-ae76-4a30-9d84-f4c3a80d63a3 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:52.388204 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:04:52.410732 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7db5aa0-ae76-4a30-9d84-f4c3a80d63a3 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:52.443663 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ab904-2e6e-43b0-a6ca-4249d8844b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:52.444310 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ab904-2e6e-43b0-a6ca-4249d8844b05 None admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:52.444598 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ab904-2e6e-43b0-a6ca-4249d8844b05 None admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:52.444874 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ab904-2e6e-43b0-a6ca-4249d8844b05 None admin] PATH_INFO: `/v3/services` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:52.445572 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ab904-2e6e-43b0-a6ca-4249d8844b05 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:04:52.445879 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ab904-2e6e-43b0-a6ca-4249d8844b05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:52.447808 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ab904-2e6e-43b0-a6ca-4249d8844b05 None admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:52.452780 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 644/1305] 10.222.0.50 () {56 vars in 1116 bytes} [Fri May 27 15:04:52 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:04:52.458501 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb90c93d-461f-44cc-be89-c7f7820580ba None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:04:52.555878 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb90c93d-461f-44cc-be89-c7f7820580ba None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:04:52.588848 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ddff6-b516-4758-9939-13231759619d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:04:52.589736 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ddff6-b516-4758-9939-13231759619d None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:04:52.590021 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ddff6-b516-4758-9939-13231759619d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:04:52.590298 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ddff6-b516-4758-9939-13231759619d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:04:52.591065 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ddff6-b516-4758-9939-13231759619d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:04:52.592419 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ddff6-b516-4758-9939-13231759619d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:04:52.610026 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 662/1306] 10.222.0.50 () {60 vars in 1144 bytes} [Fri May 27 15:04:52 2022] POST /identity/v3/endpoints => generated 331 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:05:02.449907 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=30 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:05:02.605788 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:05:56.693052 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68caa930-361b-4fb3-9922-20d2dfe9f347 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:56.694982 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68caa930-361b-4fb3-9922-20d2dfe9f347 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:56.696469 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68caa930-361b-4fb3-9922-20d2dfe9f347 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:56.703503 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 645/1307] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:05:56 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:05:56.708736 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96e765-0a16-4eb1-b0f6-40761709edee None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:56.709171 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96e765-0a16-4eb1-b0f6-40761709edee None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:56.709421 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a96e765-0a16-4eb1-b0f6-40761709edee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:56.864907 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5a96e765-0a16-4eb1-b0f6-40761709edee None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:05:56.866502 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5a96e765-0a16-4eb1-b0f6-40761709edee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:05:57.003554 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 663/1308] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:05:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:05:57.053753 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61606d0b-de12-44fd-9404-3b278c1c9e12 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:57.053970 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61606d0b-de12-44fd-9404-3b278c1c9e12 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:57.054364 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61606d0b-de12-44fd-9404-3b278c1c9e12 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:57.055755 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 646/1309] 10.222.0.50 () {54 vars in 964 bytes} [Fri May 27 15:05:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:05:57.062019 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050374e-d94b-4b3e-b4fb-4f6d615af6da None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:57.062382 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050374e-d94b-4b3e-b4fb-4f6d615af6da None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:57.062735 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f050374e-d94b-4b3e-b4fb-4f6d615af6da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:57.142964 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f050374e-d94b-4b3e-b4fb-4f6d615af6da None None] MFA Rules not processed for user `f874edc08438434189f1d14d79291487`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:05:57.144273 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f050374e-d94b-4b3e-b4fb-4f6d615af6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:05:57.173745 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:05:57.235221 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 664/1310] 10.222.0.50 () {58 vars in 1061 bytes} [Fri May 27 15:05:57 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:05:57.244700 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61606d0b-de12-44fd-9404-3b278c1c9e12 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:05:57.246555 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-61606d0b-de12-44fd-9404-3b278c1c9e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:05:57.372647 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61606d0b-de12-44fd-9404-3b278c1c9e12 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:05:57.452848 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3199e52d-809c-41dc-b529-01ac6474027d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:05:57.453504 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3199e52d-809c-41dc-b529-01ac6474027d service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:05:57.453784 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3199e52d-809c-41dc-b529-01ac6474027d service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:05:57.454056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3199e52d-809c-41dc-b529-01ac6474027d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:05:57.455764 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3199e52d-809c-41dc-b529-01ac6474027d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:05:57.512099 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:05:57.570594 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3199e52d-809c-41dc-b529-01ac6474027d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:05:57.572111 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3199e52d-809c-41dc-b529-01ac6474027d service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:05:57.592148 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 647/1311] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:05:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:06:00.152194 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51b83e4-4b75-4792-83bb-8e0ed919d7f9 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:00.152601 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51b83e4-4b75-4792-83bb-8e0ed919d7f9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:00.152677 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51b83e4-4b75-4792-83bb-8e0ed919d7f9 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:00.158341 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 665/1312] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:06:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:06:00.160139 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113e7df-d25b-4bd6-bd4c-c374ad20e02e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:00.160603 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113e7df-d25b-4bd6-bd4c-c374ad20e02e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:00.160777 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113e7df-d25b-4bd6-bd4c-c374ad20e02e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:00.236542 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5113e7df-d25b-4bd6-bd4c-c374ad20e02e None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:06:00.237767 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5113e7df-d25b-4bd6-bd4c-c374ad20e02e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:00.342610 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 648/1313] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:06:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:06:00.380679 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51b83e4-4b75-4792-83bb-8e0ed919d7f9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:06:00.382365 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e51b83e4-4b75-4792-83bb-8e0ed919d7f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:00.493116 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51b83e4-4b75-4792-83bb-8e0ed919d7f9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:06:00.573023 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:06:00.574803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb62ae4-3d53-4dba-9248-59b5d8f3d084 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:06:00.575491 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb62ae4-3d53-4dba-9248-59b5d8f3d084 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:00.576048 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb62ae4-3d53-4dba-9248-59b5d8f3d084 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:00.576411 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb62ae4-3d53-4dba-9248-59b5d8f3d084 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:00.578588 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-adb62ae4-3d53-4dba-9248-59b5d8f3d084 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:00.694565 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb62ae4-3d53-4dba-9248-59b5d8f3d084 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:06:00.696151 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb62ae4-3d53-4dba-9248-59b5d8f3d084 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:06:00.716387 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 666/1314] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:06:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:06:03.290753 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fcd019-c614-48af-92ed-f93e396dca07 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:03.291353 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fcd019-c614-48af-92ed-f93e396dca07 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:03.291884 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fcd019-c614-48af-92ed-f93e396dca07 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:03.294530 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 649/1315] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:06:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:06:03.300302 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e20854-c237-401c-b967-e1064fb8dd4d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:03.300741 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e20854-c237-401c-b967-e1064fb8dd4d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:03.300834 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e20854-c237-401c-b967-e1064fb8dd4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:03.381594 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-40e20854-c237-401c-b967-e1064fb8dd4d None None] MFA Rules not processed for user `aac9cdff3dc345eaa75a0bec09d65b4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:06:03.383519 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-40e20854-c237-401c-b967-e1064fb8dd4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:03.447792 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:06:03.497553 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 667/1316] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:06:03 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:06:03.536839 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fcd019-c614-48af-92ed-f93e396dca07 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:06:03.538317 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-18fcd019-c614-48af-92ed-f93e396dca07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:03.548690 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:06:03.646999 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fcd019-c614-48af-92ed-f93e396dca07 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:06:03.773892 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03641b2f-b116-4f30-9e89-4988d214cdf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:06:03.775613 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03641b2f-b116-4f30-9e89-4988d214cdf7 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:03.776128 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03641b2f-b116-4f30-9e89-4988d214cdf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:03.776643 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03641b2f-b116-4f30-9e89-4988d214cdf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:03.779905 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-03641b2f-b116-4f30-9e89-4988d214cdf7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:03.845089 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:06:03.892935 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03641b2f-b116-4f30-9e89-4988d214cdf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:06:03.894380 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03641b2f-b116-4f30-9e89-4988d214cdf7 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:06:03.917898 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 650/1317] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:06:03 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:06:06.818076 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585edab-5ffe-44ba-ac26-d74338a3c9bd None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:06.818658 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585edab-5ffe-44ba-ac26-d74338a3c9bd None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:06.818658 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585edab-5ffe-44ba-ac26-d74338a3c9bd None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:06.820066 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 668/1318] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:06:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:06:06.825311 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b75342-8cd8-4728-a932-f094c33bb140 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:06.825735 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b75342-8cd8-4728-a932-f094c33bb140 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:06.825842 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b75342-8cd8-4728-a932-f094c33bb140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:06.889963 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-48b75342-8cd8-4728-a932-f094c33bb140 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:06:06.891207 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-48b75342-8cd8-4728-a932-f094c33bb140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:06.986137 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 651/1319] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:06:06 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:06:07.024343 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0585edab-5ffe-44ba-ac26-d74338a3c9bd None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:06:07.026628 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0585edab-5ffe-44ba-ac26-d74338a3c9bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:07.128197 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0585edab-5ffe-44ba-ac26-d74338a3c9bd None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:06:07.200697 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf2d99c-6643-4a1c-b37f-c8be2df2ad39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:06:07.201343 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2d99c-6643-4a1c-b37f-c8be2df2ad39 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:07.201626 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2d99c-6643-4a1c-b37f-c8be2df2ad39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:07.201903 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf2d99c-6643-4a1c-b37f-c8be2df2ad39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:07.203576 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0bf2d99c-6643-4a1c-b37f-c8be2df2ad39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:07.211787 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:06:07.313220 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf2d99c-6643-4a1c-b37f-c8be2df2ad39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:06:07.314645 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf2d99c-6643-4a1c-b37f-c8be2df2ad39 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:06:07.332665 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 669/1320] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:06:07 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:06:10.205651 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305642d4-ba8e-4d2a-bc94-dbb0645a585b None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:10.206274 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305642d4-ba8e-4d2a-bc94-dbb0645a585b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:10.206274 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305642d4-ba8e-4d2a-bc94-dbb0645a585b None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:10.207799 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 652/1321] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:06:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:06:10.212791 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a3dd27-fe43-4bd3-bbf4-3d991e888be4 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:10.213205 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a3dd27-fe43-4bd3-bbf4-3d991e888be4 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:10.213356 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a3dd27-fe43-4bd3-bbf4-3d991e888be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:10.279604 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b6a3dd27-fe43-4bd3-bbf4-3d991e888be4 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:06:10.280796 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b6a3dd27-fe43-4bd3-bbf4-3d991e888be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:10.340356 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:06:10.379629 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 670/1322] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:06:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:06:10.414884 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305642d4-ba8e-4d2a-bc94-dbb0645a585b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:06:10.417331 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-305642d4-ba8e-4d2a-bc94-dbb0645a585b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:10.437156 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:06:10.522783 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305642d4-ba8e-4d2a-bc94-dbb0645a585b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:06:10.598924 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b0abc-4212-42d5-951f-bab10e2a8ee3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:06:10.599578 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b0abc-4212-42d5-951f-bab10e2a8ee3 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:10.599870 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b0abc-4212-42d5-951f-bab10e2a8ee3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:10.600150 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b0abc-4212-42d5-951f-bab10e2a8ee3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:10.601793 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d56b0abc-4212-42d5-951f-bab10e2a8ee3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:10.677730 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:06:10.718062 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b0abc-4212-42d5-951f-bab10e2a8ee3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:06:10.719698 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56b0abc-4212-42d5-951f-bab10e2a8ee3 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:06:10.738038 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 653/1323] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:06:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:06:13.008163 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2494c-72c8-4e36-8e39-635df355bc48 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:13.009243 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2494c-72c8-4e36-8e39-635df355bc48 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:13.009243 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2494c-72c8-4e36-8e39-635df355bc48 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:13.010581 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 671/1324] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:06:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:06:13.015881 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe90d85-5ca4-4422-a13e-c9471cb4a00b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:13.016292 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe90d85-5ca4-4422-a13e-c9471cb4a00b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:13.016437 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe90d85-5ca4-4422-a13e-c9471cb4a00b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:13.085002 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-afe90d85-5ca4-4422-a13e-c9471cb4a00b None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:06:13.086185 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-afe90d85-5ca4-4422-a13e-c9471cb4a00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:13.181575 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 654/1325] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:06:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:06:13.218109 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd2494c-72c8-4e36-8e39-635df355bc48 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:06:13.219703 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9cd2494c-72c8-4e36-8e39-635df355bc48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:13.320414 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd2494c-72c8-4e36-8e39-635df355bc48 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:06:13.401411 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8d38ed-4c16-4ca9-80ad-4d56bc1a68de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:06:13.402060 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8d38ed-4c16-4ca9-80ad-4d56bc1a68de service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:13.402337 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8d38ed-4c16-4ca9-80ad-4d56bc1a68de service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:13.402608 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8d38ed-4c16-4ca9-80ad-4d56bc1a68de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:13.404245 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6c8d38ed-4c16-4ca9-80ad-4d56bc1a68de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:13.422049 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:06:13.515077 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8d38ed-4c16-4ca9-80ad-4d56bc1a68de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:06:13.516505 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8d38ed-4c16-4ca9-80ad-4d56bc1a68de service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:06:13.534000 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 672/1326] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:06:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:06:15.828698 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42525a8-0bb5-4270-b196-c0bb90213103 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:15.829237 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42525a8-0bb5-4270-b196-c0bb90213103 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:15.829382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42525a8-0bb5-4270-b196-c0bb90213103 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:15.830645 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 655/1327] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:06:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:06:15.835914 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4389f5c-2dae-4a79-a4c6-8bf17eab082b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:15.836341 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4389f5c-2dae-4a79-a4c6-8bf17eab082b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:15.836474 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4389f5c-2dae-4a79-a4c6-8bf17eab082b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:15.902599 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f4389f5c-2dae-4a79-a4c6-8bf17eab082b None None] MFA Rules not processed for user `aac9cdff3dc345eaa75a0bec09d65b4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:06:15.903915 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f4389f5c-2dae-4a79-a4c6-8bf17eab082b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:15.971168 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:06:16.005468 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 673/1328] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:06:15 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:06:16.060485 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530e8f0-8b75-4339-be66-33b55bccc2f7 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:16.060791 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530e8f0-8b75-4339-be66-33b55bccc2f7 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:16.061094 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9530e8f0-8b75-4339-be66-33b55bccc2f7 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:16.062564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 656/1329] 10.222.0.50 () {54 vars in 964 bytes} [Fri May 27 15:06:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:06:16.069122 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160963e-e7d8-4c10-8216-342a0fd68b1d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:16.069480 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160963e-e7d8-4c10-8216-342a0fd68b1d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:16.069823 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a160963e-e7d8-4c10-8216-342a0fd68b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:16.146304 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a160963e-e7d8-4c10-8216-342a0fd68b1d None None] MFA Rules not processed for user `f874edc08438434189f1d14d79291487`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:06:16.147453 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a160963e-e7d8-4c10-8216-342a0fd68b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:16.224829 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 674/1330] 10.222.0.50 () {58 vars in 1061 bytes} [Fri May 27 15:06:16 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:06:16.233811 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9530e8f0-8b75-4339-be66-33b55bccc2f7 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:06:16.235403 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9530e8f0-8b75-4339-be66-33b55bccc2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:16.266678 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:06:16.341228 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9530e8f0-8b75-4339-be66-33b55bccc2f7 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:06:16.416148 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d68965f-0f11-402d-968b-846cccc91b32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:06:16.416711 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d68965f-0f11-402d-968b-846cccc91b32 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:16.416994 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d68965f-0f11-402d-968b-846cccc91b32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:16.417271 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d68965f-0f11-402d-968b-846cccc91b32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:16.418930 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3d68965f-0f11-402d-968b-846cccc91b32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:16.495482 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:06:16.528217 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d68965f-0f11-402d-968b-846cccc91b32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:06:16.529669 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d68965f-0f11-402d-968b-846cccc91b32 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:06:16.549579 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 657/1331] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:06:16 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:06:19.450668 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915abae-a618-48d5-86b1-63139c7d486a None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:19.451285 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915abae-a618-48d5-86b1-63139c7d486a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:19.451285 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5915abae-a618-48d5-86b1-63139c7d486a None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:19.452817 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 675/1332] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:06:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:06:19.458383 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a29b338-842f-4cb5-9f9e-c23beb40614a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:19.458930 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a29b338-842f-4cb5-9f9e-c23beb40614a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:19.459147 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a29b338-842f-4cb5-9f9e-c23beb40614a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:19.528663 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2a29b338-842f-4cb5-9f9e-c23beb40614a None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:06:19.529947 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2a29b338-842f-4cb5-9f9e-c23beb40614a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:19.631618 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 658/1333] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:06:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:06:19.669005 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5915abae-a618-48d5-86b1-63139c7d486a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:06:19.670634 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5915abae-a618-48d5-86b1-63139c7d486a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:19.741694 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:06:19.777512 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5915abae-a618-48d5-86b1-63139c7d486a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:06:19.853310 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba438e06-a0c7-40fd-9304-a6496f471ce4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:06:19.853906 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba438e06-a0c7-40fd-9304-a6496f471ce4 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:19.854213 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba438e06-a0c7-40fd-9304-a6496f471ce4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:19.854524 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba438e06-a0c7-40fd-9304-a6496f471ce4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:19.856290 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ba438e06-a0c7-40fd-9304-a6496f471ce4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:19.971100 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba438e06-a0c7-40fd-9304-a6496f471ce4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:06:19.972548 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba438e06-a0c7-40fd-9304-a6496f471ce4 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:06:19.983785 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:06:19.992807 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 676/1334] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:06:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:06:22.338530 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123cb887-960b-4e1d-be8d-1ab6b46aea12 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:22.339089 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123cb887-960b-4e1d-be8d-1ab6b46aea12 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:22.339220 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123cb887-960b-4e1d-be8d-1ab6b46aea12 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:22.340743 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 659/1335] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:06:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:06:22.346051 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f489265-6789-4a3e-bc99-c023b99f2129 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:22.346475 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f489265-6789-4a3e-bc99-c023b99f2129 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:22.346589 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f489265-6789-4a3e-bc99-c023b99f2129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:22.410260 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9f489265-6789-4a3e-bc99-c023b99f2129 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:06:22.411406 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9f489265-6789-4a3e-bc99-c023b99f2129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:22.506950 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 677/1336] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:06:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:06:22.547527 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123cb887-960b-4e1d-be8d-1ab6b46aea12 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:06:22.549048 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-123cb887-960b-4e1d-be8d-1ab6b46aea12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:22.588882 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:06:22.655570 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123cb887-960b-4e1d-be8d-1ab6b46aea12 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:06:22.731114 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a0feb5-443e-4d8e-b2c3-10e73c65d89c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:06:22.731766 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a0feb5-443e-4d8e-b2c3-10e73c65d89c service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:22.732053 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a0feb5-443e-4d8e-b2c3-10e73c65d89c service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:22.732331 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a0feb5-443e-4d8e-b2c3-10e73c65d89c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:22.733977 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-03a0feb5-443e-4d8e-b2c3-10e73c65d89c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:22.817305 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:06:22.844263 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a0feb5-443e-4d8e-b2c3-10e73c65d89c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:06:22.845704 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a0feb5-443e-4d8e-b2c3-10e73c65d89c service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:06:22.863720 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 660/1337] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:06:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:06:25.384377 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8616b1-e021-4d86-8b2e-c231e68fb1b9 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:25.384963 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8616b1-e021-4d86-8b2e-c231e68fb1b9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:25.384963 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8616b1-e021-4d86-8b2e-c231e68fb1b9 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:25.386413 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 678/1338] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:06:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:06:25.391891 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0229bc-55d9-43af-95c5-a3d039fc7d9a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:25.392346 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0229bc-55d9-43af-95c5-a3d039fc7d9a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:25.392449 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0229bc-55d9-43af-95c5-a3d039fc7d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:25.456447 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5e0229bc-55d9-43af-95c5-a3d039fc7d9a None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:06:25.457553 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5e0229bc-55d9-43af-95c5-a3d039fc7d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:25.553070 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 661/1339] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:06:25 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:06:25.588578 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8616b1-e021-4d86-8b2e-c231e68fb1b9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:06:25.590356 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fc8616b1-e021-4d86-8b2e-c231e68fb1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:25.664803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:06:25.691918 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8616b1-e021-4d86-8b2e-c231e68fb1b9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:06:25.763687 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d95182-6125-4b99-921f-c764f527559d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:06:25.764256 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d95182-6125-4b99-921f-c764f527559d service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:25.764537 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d95182-6125-4b99-921f-c764f527559d service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:25.764810 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d95182-6125-4b99-921f-c764f527559d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:25.766420 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c7d95182-6125-4b99-921f-c764f527559d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:25.872474 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d95182-6125-4b99-921f-c764f527559d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:06:25.873892 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d95182-6125-4b99-921f-c764f527559d service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:06:25.894019 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 679/1340] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:06:25 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:06:30.157187 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed7782-88b2-4c6c-a20b-1af04873f06f None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:30.157187 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed7782-88b2-4c6c-a20b-1af04873f06f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:30.157187 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed7782-88b2-4c6c-a20b-1af04873f06f None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:30.158884 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 662/1341] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:06:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:06:30.163400 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92508b83-dda7-48c6-9a93-8d073697c393 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:30.163813 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92508b83-dda7-48c6-9a93-8d073697c393 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:30.163967 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92508b83-dda7-48c6-9a93-8d073697c393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:30.170618 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:06:30.235822 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-92508b83-dda7-48c6-9a93-8d073697c393 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:06:30.237013 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-92508b83-dda7-48c6-9a93-8d073697c393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:30.342330 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 680/1342] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:06:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:06:30.381167 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed7782-88b2-4c6c-a20b-1af04873f06f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:06:30.383440 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-63ed7782-88b2-4c6c-a20b-1af04873f06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:30.435209 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:06:30.480627 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed7782-88b2-4c6c-a20b-1af04873f06f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:06:30.484185 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409f82db-ae90-499a-a646-aa69c8d38661 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:06:30.484836 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f82db-ae90-499a-a646-aa69c8d38661 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:30.485118 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f82db-ae90-499a-a646-aa69c8d38661 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:30.485394 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f82db-ae90-499a-a646-aa69c8d38661 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:30.495041 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409f82db-ae90-499a-a646-aa69c8d38661 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:06:30.496487 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409f82db-ae90-499a-a646-aa69c8d38661 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:06:30.505846 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 663/1343] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:06:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:06:33.351160 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6fc720-9649-4010-b2a7-1ca304d7a80c None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:33.351735 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6fc720-9649-4010-b2a7-1ca304d7a80c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:33.351735 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6fc720-9649-4010-b2a7-1ca304d7a80c None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:33.353629 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 681/1344] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:06:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:06:33.359137 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c42499-cb09-424f-92e7-cb7c920e2843 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:33.359550 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c42499-cb09-424f-92e7-cb7c920e2843 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:33.359700 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c42499-cb09-424f-92e7-cb7c920e2843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:33.387496 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e3c42499-cb09-424f-92e7-cb7c920e2843 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:06:33.388951 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e3c42499-cb09-424f-92e7-cb7c920e2843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:33.466350 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 664/1345] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:06:33 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:06:33.503862 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6fc720-9649-4010-b2a7-1ca304d7a80c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:06:33.514616 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6fc720-9649-4010-b2a7-1ca304d7a80c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:06:33.518154 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b588f71-55cd-4f91-99b2-031e6271749d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:06:33.518791 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b588f71-55cd-4f91-99b2-031e6271749d service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:33.519105 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b588f71-55cd-4f91-99b2-031e6271749d service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:33.519387 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b588f71-55cd-4f91-99b2-031e6271749d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:33.528992 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b588f71-55cd-4f91-99b2-031e6271749d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:06:33.530427 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b588f71-55cd-4f91-99b2-031e6271749d service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:06:33.540194 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 682/1346] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:06:33 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:06:37.727270 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea7645-76eb-4883-9e8a-6e1bf1431099 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:37.727853 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea7645-76eb-4883-9e8a-6e1bf1431099 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:37.728125 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea7645-76eb-4883-9e8a-6e1bf1431099 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:37.729538 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 665/1347] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:06:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:06:37.734795 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85a2c1-070c-48f1-8d3e-ed33ab142eed None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:37.735232 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85a2c1-070c-48f1-8d3e-ed33ab142eed None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:37.735383 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf85a2c1-070c-48f1-8d3e-ed33ab142eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:37.753821 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-bf85a2c1-070c-48f1-8d3e-ed33ab142eed None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:06:37.755103 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bf85a2c1-070c-48f1-8d3e-ed33ab142eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:37.762834 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 683/1348] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:06:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:06:37.799607 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ea7645-76eb-4883-9e8a-6e1bf1431099 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:06:37.810678 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ea7645-76eb-4883-9e8a-6e1bf1431099 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:06:37.814121 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206c275c-a9c8-4170-b645-c46d711f10ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:06:37.814751 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c275c-a9c8-4170-b645-c46d711f10ad service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:37.815065 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c275c-a9c8-4170-b645-c46d711f10ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:37.815342 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c275c-a9c8-4170-b645-c46d711f10ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:37.824807 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206c275c-a9c8-4170-b645-c46d711f10ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:06:37.826262 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206c275c-a9c8-4170-b645-c46d711f10ad service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:06:37.835429 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 666/1349] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:06:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:06:43.332932 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c633b8-4ef9-4a94-b0df-bfc0894fbf9a None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:43.333849 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c633b8-4ef9-4a94-b0df-bfc0894fbf9a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:43.334347 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c633b8-4ef9-4a94-b0df-bfc0894fbf9a None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:43.336910 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 684/1350] 10.222.0.50 () {54 vars in 921 bytes} [Fri May 27 15:06:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:06:43.342071 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94557ba2-b527-4881-9ec5-3d3a898eb48b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:43.343148 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94557ba2-b527-4881-9ec5-3d3a898eb48b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:43.343627 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94557ba2-b527-4881-9ec5-3d3a898eb48b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:43.425352 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-94557ba2-b527-4881-9ec5-3d3a898eb48b None None] MFA Rules not processed for user `8293dab3ffd647f5927d323572fdbdce`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:06:43.426551 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-94557ba2-b527-4881-9ec5-3d3a898eb48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:43.473270 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:06:43.567347 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 667/1351] 10.222.0.50 () {58 vars in 1018 bytes} [Fri May 27 15:06:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:06:47.625652 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4baccc-11be-45cd-86dc-9f2ac7fb9785 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:47.626181 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4baccc-11be-45cd-86dc-9f2ac7fb9785 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:47.626181 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4baccc-11be-45cd-86dc-9f2ac7fb9785 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:47.627790 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 685/1352] 10.222.0.50 () {54 vars in 921 bytes} [Fri May 27 15:06:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:06:47.632931 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3251f-94dd-4ff3-9566-6d09ce6402c9 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:47.633476 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3251f-94dd-4ff3-9566-6d09ce6402c9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:47.633568 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3251f-94dd-4ff3-9566-6d09ce6402c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:47.652291 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-04a3251f-94dd-4ff3-9566-6d09ce6402c9 None None] MFA Rules not processed for user `8293dab3ffd647f5927d323572fdbdce`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:06:47.653582 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-04a3251f-94dd-4ff3-9566-6d09ce6402c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:47.662381 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 668/1353] 10.222.0.50 () {58 vars in 1018 bytes} [Fri May 27 15:06:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:06:47.748718 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=72 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:06:49.244408 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f288d8ad-81cc-4f92-bb42-1b40bcb3e616 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:49.245061 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f288d8ad-81cc-4f92-bb42-1b40bcb3e616 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:49.245061 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f288d8ad-81cc-4f92-bb42-1b40bcb3e616 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:49.246375 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 686/1354] 10.222.0.50 () {54 vars in 921 bytes} [Fri May 27 15:06:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:06:49.251449 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91befd49-afa4-402b-b216-07fe2132c31a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:49.251861 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91befd49-afa4-402b-b216-07fe2132c31a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:49.252142 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91befd49-afa4-402b-b216-07fe2132c31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:49.271394 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-91befd49-afa4-402b-b216-07fe2132c31a None None] MFA Rules not processed for user `8293dab3ffd647f5927d323572fdbdce`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:06:49.272598 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-91befd49-afa4-402b-b216-07fe2132c31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:49.281492 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 669/1355] 10.222.0.50 () {58 vars in 1018 bytes} [Fri May 27 15:06:49 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:06:53.326442 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e7326-9d10-4002-ad9f-b3db3d7fa0ab None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:53.327041 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e7326-9d10-4002-ad9f-b3db3d7fa0ab None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:53.327041 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e7326-9d10-4002-ad9f-b3db3d7fa0ab None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:53.328389 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 687/1356] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:06:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:06:53.333959 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b5518-0a46-4197-9dd7-3a4dcc2c68ae None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:53.334393 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b5518-0a46-4197-9dd7-3a4dcc2c68ae None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:53.334708 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b5518-0a46-4197-9dd7-3a4dcc2c68ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:53.354131 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f52b5518-0a46-4197-9dd7-3a4dcc2c68ae None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:06:53.355521 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f52b5518-0a46-4197-9dd7-3a4dcc2c68ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:53.363446 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 670/1357] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:06:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:06:53.419950 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1723fc-b446-4ba3-a616-261f2517fdaa None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:53.421112 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1723fc-b446-4ba3-a616-261f2517fdaa None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:53.421670 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1723fc-b446-4ba3-a616-261f2517fdaa None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:53.423793 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 688/1358] 10.222.0.50 () {54 vars in 961 bytes} [Fri May 27 15:06:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:06:53.432911 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d776291-2b3e-491e-95d1-5d38ab954c57 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:53.433522 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d776291-2b3e-491e-95d1-5d38ab954c57 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:53.434455 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d776291-2b3e-491e-95d1-5d38ab954c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:53.488082 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2d776291-2b3e-491e-95d1-5d38ab954c57 None None] MFA Rules not processed for user `451a9232f95e494683d81b6078a318d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:06:53.489430 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2d776291-2b3e-491e-95d1-5d38ab954c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:53.574718 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 671/1359] 10.222.0.50 () {58 vars in 1058 bytes} [Fri May 27 15:06:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:06:53.582475 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1723fc-b446-4ba3-a616-261f2517fdaa None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:06:53.597676 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1723fc-b446-4ba3-a616-261f2517fdaa None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:06:53.604541 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7e0eac-be9b-462c-94ed-c0fae5bfe1bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '451a9232f95e494683d81b6078a318d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:06:53.605466 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e0eac-be9b-462c-94ed-c0fae5bfe1bb service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:53.605956 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e0eac-be9b-462c-94ed-c0fae5bfe1bb service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:53.606472 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e0eac-be9b-462c-94ed-c0fae5bfe1bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:53.616979 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7e0eac-be9b-462c-94ed-c0fae5bfe1bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:06:53.619305 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7e0eac-be9b-462c-94ed-c0fae5bfe1bb service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:06:53.629199 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 689/1360] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:06:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:06:56.090276 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf94e6-738d-4c84-8b11-19b7366da2e5 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:56.090805 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf94e6-738d-4c84-8b11-19b7366da2e5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:56.090805 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf94e6-738d-4c84-8b11-19b7366da2e5 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:56.092198 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 672/1361] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:06:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:06:56.097534 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2632a59-4791-41a4-bd93-60a51997508a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:56.098779 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2632a59-4791-41a4-bd93-60a51997508a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:56.099458 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2632a59-4791-41a4-bd93-60a51997508a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:56.121781 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f2632a59-4791-41a4-bd93-60a51997508a None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:06:56.123062 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f2632a59-4791-41a4-bd93-60a51997508a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:56.131868 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 690/1362] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:06:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:06:56.176901 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bf94e6-738d-4c84-8b11-19b7366da2e5 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:06:56.191682 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bf94e6-738d-4c84-8b11-19b7366da2e5 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:06:56.197994 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6678a41a-3d50-4dd9-8d55-32259bcb0c02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '451a9232f95e494683d81b6078a318d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:06:56.198635 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6678a41a-3d50-4dd9-8d55-32259bcb0c02 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:56.198950 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6678a41a-3d50-4dd9-8d55-32259bcb0c02 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:56.199241 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6678a41a-3d50-4dd9-8d55-32259bcb0c02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:56.207951 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6678a41a-3d50-4dd9-8d55-32259bcb0c02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:06:56.209554 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6678a41a-3d50-4dd9-8d55-32259bcb0c02 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:06:56.220083 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 673/1363] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:06:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:06:58.857453 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42507e-c740-4c6d-97eb-2a38d0fdafd4 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:58.858438 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42507e-c740-4c6d-97eb-2a38d0fdafd4 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:58.858956 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42507e-c740-4c6d-97eb-2a38d0fdafd4 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:58.861581 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 691/1364] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:06:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 15:06:58.866088 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbc60f-86cd-4ecc-95bb-33743385adce None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:58.866531 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbc60f-86cd-4ecc-95bb-33743385adce None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:58.866645 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbc60f-86cd-4ecc-95bb-33743385adce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:58.885229 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ddbbc60f-86cd-4ecc-95bb-33743385adce None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:06:58.886421 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ddbbc60f-86cd-4ecc-95bb-33743385adce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:58.894545 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 674/1365] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:06:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:06:58.950179 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67abd8-7b24-49e9-b930-0360ff2d9e23 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:58.950464 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67abd8-7b24-49e9-b930-0360ff2d9e23 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:58.950999 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67abd8-7b24-49e9-b930-0360ff2d9e23 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:58.952602 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 692/1366] 10.222.0.50 () {54 vars in 961 bytes} [Fri May 27 15:06:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:06:58.959464 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bf8d70-123f-45ee-a942-c55a702518b1 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:58.959758 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bf8d70-123f-45ee-a942-c55a702518b1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:58.960027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bf8d70-123f-45ee-a942-c55a702518b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:58.984206 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b2bf8d70-123f-45ee-a942-c55a702518b1 None None] MFA Rules not processed for user `451a9232f95e494683d81b6078a318d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:06:58.985380 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b2bf8d70-123f-45ee-a942-c55a702518b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:06:58.995368 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 675/1367] 10.222.0.50 () {58 vars in 1058 bytes} [Fri May 27 15:06:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:06:59.003420 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db67abd8-7b24-49e9-b930-0360ff2d9e23 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:06:59.018435 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db67abd8-7b24-49e9-b930-0360ff2d9e23 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:06:59.024096 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc791185-b13e-4078-8f0f-186763b81bdd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '451a9232f95e494683d81b6078a318d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:06:59.024736 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc791185-b13e-4078-8f0f-186763b81bdd service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:06:59.025071 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc791185-b13e-4078-8f0f-186763b81bdd service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:06:59.025354 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc791185-b13e-4078-8f0f-186763b81bdd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:06:59.033449 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc791185-b13e-4078-8f0f-186763b81bdd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:06:59.035052 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc791185-b13e-4078-8f0f-186763b81bdd service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:06:59.045029 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 693/1368] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:06:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:07:01.329924 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3fa56-2e44-4822-abfe-237c9a6be4f7 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:01.330483 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3fa56-2e44-4822-abfe-237c9a6be4f7 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:01.330483 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db3fa56-2e44-4822-abfe-237c9a6be4f7 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:01.331865 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 676/1369] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:07:01.338032 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47604fd5-4a23-4c54-8b3c-04e308655c02 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:01.338488 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47604fd5-4a23-4c54-8b3c-04e308655c02 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:01.338584 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47604fd5-4a23-4c54-8b3c-04e308655c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:01.357216 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-47604fd5-4a23-4c54-8b3c-04e308655c02 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:07:01.358396 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-47604fd5-4a23-4c54-8b3c-04e308655c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:07:01.366368 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 694/1370] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:07:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:07:01.410225 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db3fa56-2e44-4822-abfe-237c9a6be4f7 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:07:01.426879 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db3fa56-2e44-4822-abfe-237c9a6be4f7 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:07:01.432777 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e56a68d-c881-4d64-9998-9444c15dbca0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '451a9232f95e494683d81b6078a318d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:07:01.433511 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56a68d-c881-4d64-9998-9444c15dbca0 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:01.433871 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56a68d-c881-4d64-9998-9444c15dbca0 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:01.434224 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56a68d-c881-4d64-9998-9444c15dbca0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:01.443041 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e56a68d-c881-4d64-9998-9444c15dbca0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:07:01.444643 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e56a68d-c881-4d64-9998-9444c15dbca0 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:07:01.455210 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 677/1371] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:07:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:07:01.699426 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34afdff5-2d12-42c0-b7b1-2bd87cc77738 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:01.699639 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34afdff5-2d12-42c0-b7b1-2bd87cc77738 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:01.699942 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34afdff5-2d12-42c0-b7b1-2bd87cc77738 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:01.701471 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 695/1372] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:07:01.707980 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28518236-8953-49a1-80f0-c5acc01cbe70 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:01.708320 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28518236-8953-49a1-80f0-c5acc01cbe70 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:01.708547 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28518236-8953-49a1-80f0-c5acc01cbe70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:01.728880 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-28518236-8953-49a1-80f0-c5acc01cbe70 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:07:01.730090 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-28518236-8953-49a1-80f0-c5acc01cbe70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:07:01.738902 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 678/1373] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:07:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:07:02.047685 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27752a-8a5a-4aeb-9fe6-f1cdfd8bebd1 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:02.047979 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27752a-8a5a-4aeb-9fe6-f1cdfd8bebd1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:02.048265 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27752a-8a5a-4aeb-9fe6-f1cdfd8bebd1 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:02.049899 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 696/1374] 10.222.0.50 () {54 vars in 963 bytes} [Fri May 27 15:07:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 15:07:02.056946 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a3da79-4165-4a1e-aef7-7e3ff6221a97 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:02.056946 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a3da79-4165-4a1e-aef7-7e3ff6221a97 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:02.057334 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a3da79-4165-4a1e-aef7-7e3ff6221a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:02.064795 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:07:02.109255 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a0a3da79-4165-4a1e-aef7-7e3ff6221a97 None None] MFA Rules not processed for user `896af6ccf89847e5903f6a4fe3d9d810`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:07:02.110453 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a0a3da79-4165-4a1e-aef7-7e3ff6221a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:07:02.184144 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 679/1375] 10.222.0.50 () {58 vars in 1060 bytes} [Fri May 27 15:07:02 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:07:02.191765 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f27752a-8a5a-4aeb-9fe6-f1cdfd8bebd1 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:07:02.204094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f27752a-8a5a-4aeb-9fe6-f1cdfd8bebd1 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:07:02.207710 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d27ca8-1cff-4e23-9f65-2998ce57fe0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:07:02.208413 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d27ca8-1cff-4e23-9f65-2998ce57fe0e service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:02.208705 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d27ca8-1cff-4e23-9f65-2998ce57fe0e service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:02.208985 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d27ca8-1cff-4e23-9f65-2998ce57fe0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:02.219018 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d27ca8-1cff-4e23-9f65-2998ce57fe0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:07:02.220609 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d27ca8-1cff-4e23-9f65-2998ce57fe0e service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:07:02.234826 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 697/1376] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:07:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:07:03.729487 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef785480-7b02-4596-9411-5248185684e7 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:03.730041 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef785480-7b02-4596-9411-5248185684e7 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:03.730041 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef785480-7b02-4596-9411-5248185684e7 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:03.731428 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 680/1377] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:07:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:07:03.736985 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a974529-3216-4d0e-a242-642780a4d443 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:03.737434 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a974529-3216-4d0e-a242-642780a4d443 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:03.737594 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a974529-3216-4d0e-a242-642780a4d443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:03.755911 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9a974529-3216-4d0e-a242-642780a4d443 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:07:03.757079 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9a974529-3216-4d0e-a242-642780a4d443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:07:03.765100 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 698/1378] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:07:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:07:03.811906 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef785480-7b02-4596-9411-5248185684e7 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:07:03.827000 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef785480-7b02-4596-9411-5248185684e7 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:07:03.833045 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82c0eca-c82c-4d4d-892d-b435b14faffb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '451a9232f95e494683d81b6078a318d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:07:03.833718 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c0eca-c82c-4d4d-892d-b435b14faffb service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:03.834005 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c0eca-c82c-4d4d-892d-b435b14faffb service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:03.834285 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c0eca-c82c-4d4d-892d-b435b14faffb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:03.844650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82c0eca-c82c-4d4d-892d-b435b14faffb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:07:03.846298 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82c0eca-c82c-4d4d-892d-b435b14faffb service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:07:03.858356 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 681/1379] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:07:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:07:06.138068 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d584c1c-4e72-4f10-8f85-f6ebff362ed6 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:06.138577 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d584c1c-4e72-4f10-8f85-f6ebff362ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:06.138577 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d584c1c-4e72-4f10-8f85-f6ebff362ed6 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:06.140225 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 699/1380] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:07:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:07:06.145415 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a370e8fa-aa90-40c2-bc4a-e60c42c4dafd None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:06.145858 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a370e8fa-aa90-40c2-bc4a-e60c42c4dafd None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:06.145992 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a370e8fa-aa90-40c2-bc4a-e60c42c4dafd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:06.169584 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a370e8fa-aa90-40c2-bc4a-e60c42c4dafd None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:07:06.171037 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a370e8fa-aa90-40c2-bc4a-e60c42c4dafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:07:06.180374 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 682/1381] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:07:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:07:06.223638 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d584c1c-4e72-4f10-8f85-f6ebff362ed6 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:07:06.238682 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d584c1c-4e72-4f10-8f85-f6ebff362ed6 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:07:06.244950 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dc3421-7c63-431d-b305-329893081109 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '451a9232f95e494683d81b6078a318d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:07:06.245592 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc3421-7c63-431d-b305-329893081109 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:06.245892 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc3421-7c63-431d-b305-329893081109 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:06.246150 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dc3421-7c63-431d-b305-329893081109 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:06.254979 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dc3421-7c63-431d-b305-329893081109 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:07:06.256772 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dc3421-7c63-431d-b305-329893081109 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:07:06.266907 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 700/1382] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:07:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:07:08.673672 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288eb977-b3bf-422c-a9c3-adec046ffe2e None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:08.674145 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288eb977-b3bf-422c-a9c3-adec046ffe2e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:08.674240 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288eb977-b3bf-422c-a9c3-adec046ffe2e None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:08.675768 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 683/1383] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:07:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:07:08.681663 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21736e41-64e4-4942-bf6a-8aee4126a505 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:08.682097 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21736e41-64e4-4942-bf6a-8aee4126a505 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:08.682222 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21736e41-64e4-4942-bf6a-8aee4126a505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:08.702761 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-21736e41-64e4-4942-bf6a-8aee4126a505 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:07:08.703970 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-21736e41-64e4-4942-bf6a-8aee4126a505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:07:08.712206 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 701/1384] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:07:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:07:08.754052 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288eb977-b3bf-422c-a9c3-adec046ffe2e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:07:08.768336 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288eb977-b3bf-422c-a9c3-adec046ffe2e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:07:08.774411 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a94e7d-67d6-42ef-b9db-38d9ccfd7dc7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '451a9232f95e494683d81b6078a318d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:07:08.775077 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a94e7d-67d6-42ef-b9db-38d9ccfd7dc7 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:08.775363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a94e7d-67d6-42ef-b9db-38d9ccfd7dc7 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:08.775640 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a94e7d-67d6-42ef-b9db-38d9ccfd7dc7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:08.783975 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a94e7d-67d6-42ef-b9db-38d9ccfd7dc7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:07:08.785553 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a94e7d-67d6-42ef-b9db-38d9ccfd7dc7 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:07:08.796071 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 684/1385] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:07:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:07:11.044987 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c636f02-f2d2-426d-b012-43a058791b92 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:11.045579 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c636f02-f2d2-426d-b012-43a058791b92 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:11.045696 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c636f02-f2d2-426d-b012-43a058791b92 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:11.047519 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 702/1386] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:07:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:07:11.052792 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b2675-dcf2-4e96-a091-460fe4d27443 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:11.053204 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b2675-dcf2-4e96-a091-460fe4d27443 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:11.053380 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b2675-dcf2-4e96-a091-460fe4d27443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:11.072690 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ec6b2675-dcf2-4e96-a091-460fe4d27443 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:07:11.073957 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ec6b2675-dcf2-4e96-a091-460fe4d27443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:07:11.081961 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 685/1387] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:07:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:07:11.137340 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c636f02-f2d2-426d-b012-43a058791b92 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:07:11.155882 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c636f02-f2d2-426d-b012-43a058791b92 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:07:11.161821 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b982a899-ddcd-43b4-a841-7fb58ee8094a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '451a9232f95e494683d81b6078a318d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:07:11.162511 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b982a899-ddcd-43b4-a841-7fb58ee8094a service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:11.162837 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b982a899-ddcd-43b4-a841-7fb58ee8094a service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:11.163146 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b982a899-ddcd-43b4-a841-7fb58ee8094a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:11.173759 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b982a899-ddcd-43b4-a841-7fb58ee8094a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:07:11.175849 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b982a899-ddcd-43b4-a841-7fb58ee8094a service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:07:11.188032 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 703/1388] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:07:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:07:13.865760 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d1fcb-b7e0-40f3-8526-160ac7c59156 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:13.866345 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d1fcb-b7e0-40f3-8526-160ac7c59156 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:13.866345 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d1fcb-b7e0-40f3-8526-160ac7c59156 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:13.868286 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 686/1389] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:07:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:07:13.875689 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04efb44b-7b8c-4d3c-9342-f030d2b1ffb5 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:13.876142 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04efb44b-7b8c-4d3c-9342-f030d2b1ffb5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:13.876473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04efb44b-7b8c-4d3c-9342-f030d2b1ffb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:13.900022 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-04efb44b-7b8c-4d3c-9342-f030d2b1ffb5 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:07:13.901210 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-04efb44b-7b8c-4d3c-9342-f030d2b1ffb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:07:13.909601 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 704/1390] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:07:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:07:13.955610 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d1fcb-b7e0-40f3-8526-160ac7c59156 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:07:13.974105 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d1fcb-b7e0-40f3-8526-160ac7c59156 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:07:13.981933 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ff106e-cc75-4862-9768-6fe954eae93e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '451a9232f95e494683d81b6078a318d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:07:13.982613 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff106e-cc75-4862-9768-6fe954eae93e service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:13.982944 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff106e-cc75-4862-9768-6fe954eae93e service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:13.983222 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ff106e-cc75-4862-9768-6fe954eae93e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:13.994723 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ff106e-cc75-4862-9768-6fe954eae93e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:07:13.996380 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ff106e-cc75-4862-9768-6fe954eae93e service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:07:14.005789 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 687/1391] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:07:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:07:16.338076 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d7972-2ee2-48a3-bbd7-9e5dba0d1fba None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:16.338689 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d7972-2ee2-48a3-bbd7-9e5dba0d1fba None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:16.338689 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31d7972-2ee2-48a3-bbd7-9e5dba0d1fba None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:16.340516 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 705/1392] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:07:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:07:16.345643 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e2ea4f-f73f-4ab9-b9c2-2c2d9a1d50d8 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:16.346070 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e2ea4f-f73f-4ab9-b9c2-2c2d9a1d50d8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:16.346333 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e2ea4f-f73f-4ab9-b9c2-2c2d9a1d50d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:16.364433 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-21e2ea4f-f73f-4ab9-b9c2-2c2d9a1d50d8 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:07:16.365617 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-21e2ea4f-f73f-4ab9-b9c2-2c2d9a1d50d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:07:16.374153 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 688/1393] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:07:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:07:16.418101 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31d7972-2ee2-48a3-bbd7-9e5dba0d1fba None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:07:16.432964 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31d7972-2ee2-48a3-bbd7-9e5dba0d1fba None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:07:16.438387 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94950e56-e3d5-4e0f-a6fa-65908563de80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '451a9232f95e494683d81b6078a318d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:07:16.439076 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94950e56-e3d5-4e0f-a6fa-65908563de80 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:16.439363 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94950e56-e3d5-4e0f-a6fa-65908563de80 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:16.439643 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94950e56-e3d5-4e0f-a6fa-65908563de80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:16.448568 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94950e56-e3d5-4e0f-a6fa-65908563de80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:07:16.450158 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94950e56-e3d5-4e0f-a6fa-65908563de80 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:07:16.459708 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 706/1394] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:07:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:07:18.200353 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285bf15e-1ca5-4af9-b203-daf5cad30005 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:18.200908 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285bf15e-1ca5-4af9-b203-daf5cad30005 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:18.200908 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285bf15e-1ca5-4af9-b203-daf5cad30005 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:18.202306 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 689/1395] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:07:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:07:18.207651 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b154f4c-ca33-4c60-bc26-96c9b8f4a1d8 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:18.208091 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b154f4c-ca33-4c60-bc26-96c9b8f4a1d8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:18.208195 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b154f4c-ca33-4c60-bc26-96c9b8f4a1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:18.215276 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=52 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:07:18.227544 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8b154f4c-ca33-4c60-bc26-96c9b8f4a1d8 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:07:18.228730 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8b154f4c-ca33-4c60-bc26-96c9b8f4a1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:07:18.237533 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 707/1396] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:07:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:07:18.292127 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c2da7-1bd2-419e-8e7d-24b086620aec None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:18.292444 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c2da7-1bd2-419e-8e7d-24b086620aec None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:18.292729 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c2da7-1bd2-419e-8e7d-24b086620aec None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:18.294182 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 690/1397] 10.222.0.50 () {54 vars in 962 bytes} [Fri May 27 15:07:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:07:18.299262 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b151e4-4575-4931-b3f7-da2a83b8ff13 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:18.299625 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b151e4-4575-4931-b3f7-da2a83b8ff13 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:18.299972 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b151e4-4575-4931-b3f7-da2a83b8ff13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:18.350014 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-23b151e4-4575-4931-b3f7-da2a83b8ff13 None None] MFA Rules not processed for user `1afda9d6b1d64e6ab400b26124cf1290`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:07:18.351328 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-23b151e4-4575-4931-b3f7-da2a83b8ff13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:07:18.411505 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 708/1398] 10.222.0.50 () {58 vars in 1059 bytes} [Fri May 27 15:07:18 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:07:18.419341 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c2da7-1bd2-419e-8e7d-24b086620aec None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:07:18.430072 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c2da7-1bd2-419e-8e7d-24b086620aec None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:07:18.433570 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f0377b-daf7-4951-99c9-e301f10af016 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1afda9d6b1d64e6ab400b26124cf1290', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:07:18.434196 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0377b-daf7-4951-99c9-e301f10af016 service glance] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:18.434479 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0377b-daf7-4951-99c9-e301f10af016 service glance] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:18.434754 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0377b-daf7-4951-99c9-e301f10af016 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:18.443546 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f0377b-daf7-4951-99c9-e301f10af016 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:07:18.445740 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f0377b-daf7-4951-99c9-e301f10af016 service glance] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:07:18.455030 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 691/1399] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:07:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:07:18.709476 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb9ccd-3c84-440d-839a-64ead202303f None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:18.709688 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb9ccd-3c84-440d-839a-64ead202303f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:18.710581 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfb9ccd-3c84-440d-839a-64ead202303f None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:18.712327 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 709/1400] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:07:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:07:18.717643 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cbece2-dc7e-401d-b0b8-0bf4d061a03c None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:18.717981 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cbece2-dc7e-401d-b0b8-0bf4d061a03c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:18.718258 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cbece2-dc7e-401d-b0b8-0bf4d061a03c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:18.736993 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-82cbece2-dc7e-401d-b0b8-0bf4d061a03c None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:07:18.738184 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-82cbece2-dc7e-401d-b0b8-0bf4d061a03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:07:18.746409 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 692/1401] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:07:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:07:18.791347 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfb9ccd-3c84-440d-839a-64ead202303f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:07:18.805918 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfb9ccd-3c84-440d-839a-64ead202303f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:07:18.811294 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc21e7-b355-4e1b-ab0c-674b8631ab18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '451a9232f95e494683d81b6078a318d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:07:18.811926 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc21e7-b355-4e1b-ab0c-674b8631ab18 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:18.812211 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc21e7-b355-4e1b-ab0c-674b8631ab18 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:18.812491 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc21e7-b355-4e1b-ab0c-674b8631ab18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:18.820764 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fc21e7-b355-4e1b-ab0c-674b8631ab18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:07:18.822339 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fc21e7-b355-4e1b-ab0c-674b8631ab18 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:07:18.834755 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 710/1402] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:07:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:07:19.013048 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f91d54-391f-423f-8fe3-6c08ca495908 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:19.013334 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f91d54-391f-423f-8fe3-6c08ca495908 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:19.013611 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f91d54-391f-423f-8fe3-6c08ca495908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:19.068046 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-26f91d54-391f-423f-8fe3-6c08ca495908 None None] MFA Rules not processed for user `09c23d1c732c452a9597c647ad0241f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:07:19.069273 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-26f91d54-391f-423f-8fe3-6c08ca495908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:07:19.109208 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:07:19.142830 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 693/1403] 10.222.0.50 () {58 vars in 1009 bytes} [Fri May 27 15:07:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:07:19.187065 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cefab2b-b66e-4dec-8124-56e0c5d434da None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:19.187362 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cefab2b-b66e-4dec-8124-56e0c5d434da None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:19.187644 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cefab2b-b66e-4dec-8124-56e0c5d434da None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:19.189062 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 711/1404] 10.222.0.50 () {54 vars in 942 bytes} [Fri May 27 15:07:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:07:19.195887 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b57d59d-7ebe-469f-8ed0-dc7d200230ba None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:19.196195 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b57d59d-7ebe-469f-8ed0-dc7d200230ba None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:19.196473 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b57d59d-7ebe-469f-8ed0-dc7d200230ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:19.245529 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3b57d59d-7ebe-469f-8ed0-dc7d200230ba None None] MFA Rules not processed for user `18903f2eab7c415096e075d47f0b9a4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:07:19.246775 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3b57d59d-7ebe-469f-8ed0-dc7d200230ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:07:19.317310 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 694/1405] 10.222.0.50 () {58 vars in 1039 bytes} [Fri May 27 15:07:19 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:07:19.327307 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cefab2b-b66e-4dec-8124-56e0c5d434da None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:07:19.341788 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cefab2b-b66e-4dec-8124-56e0c5d434da None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:07:19.346971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aeca37-c8bc-4dc2-8e70-c2b5ac259f21 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18903f2eab7c415096e075d47f0b9a4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:07:19.347734 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aeca37-c8bc-4dc2-8e70-c2b5ac259f21 service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:19.348022 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aeca37-c8bc-4dc2-8e70-c2b5ac259f21 service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:19.348295 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aeca37-c8bc-4dc2-8e70-c2b5ac259f21 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:19.357208 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aeca37-c8bc-4dc2-8e70-c2b5ac259f21 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:07:19.358813 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aeca37-c8bc-4dc2-8e70-c2b5ac259f21 service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:07:19.367102 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 712/1406] 10.222.0.50 () {60 vars in 1384 bytes} [Fri May 27 15:07:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 15:07:21.176754 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744f44b-ba07-402b-bfee-ee60aa3a820e None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:21.177585 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744f44b-ba07-402b-bfee-ee60aa3a820e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:21.177585 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d744f44b-ba07-402b-bfee-ee60aa3a820e None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:21.187164 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 695/1407] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:07:21 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:07:21.191658 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbde3f1-10a6-4f0f-b2c5-2f2bfddae36a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:21.193211 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbde3f1-10a6-4f0f-b2c5-2f2bfddae36a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:21.193504 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbde3f1-10a6-4f0f-b2c5-2f2bfddae36a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:21.221164 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-cdbde3f1-10a6-4f0f-b2c5-2f2bfddae36a None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:07:21.223014 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cdbde3f1-10a6-4f0f-b2c5-2f2bfddae36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:07:21.235267 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 713/1408] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:07:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:07:21.293449 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d744f44b-ba07-402b-bfee-ee60aa3a820e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:07:21.311218 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d744f44b-ba07-402b-bfee-ee60aa3a820e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:07:21.316845 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca0e90a-d4bf-4f98-b17e-ead8d5e91d9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '451a9232f95e494683d81b6078a318d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:07:21.317492 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0e90a-d4bf-4f98-b17e-ead8d5e91d9b service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:21.317773 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0e90a-d4bf-4f98-b17e-ead8d5e91d9b service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:21.318047 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca0e90a-d4bf-4f98-b17e-ead8d5e91d9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:21.327125 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca0e90a-d4bf-4f98-b17e-ead8d5e91d9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:07:21.328718 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca0e90a-d4bf-4f98-b17e-ead8d5e91d9b service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:07:21.338245 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 696/1409] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:07:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:07:31.213245 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=55 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:07:31.333842 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=62 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:07:42.716323 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d06717-0cf9-4042-b529-d2498a54ff5b None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:42.717079 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d06717-0cf9-4042-b529-d2498a54ff5b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:42.717079 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d06717-0cf9-4042-b529-d2498a54ff5b None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:42.718260 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 714/1410] 10.222.0.50 () {54 vars in 918 bytes} [Fri May 27 15:07:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:07:42.723349 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cceb5d5-063f-4cb0-8cb5-c30b13dee367 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:42.723823 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cceb5d5-063f-4cb0-8cb5-c30b13dee367 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:42.723930 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cceb5d5-063f-4cb0-8cb5-c30b13dee367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:42.742868 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3cceb5d5-063f-4cb0-8cb5-c30b13dee367 None None] MFA Rules not processed for user `8293dab3ffd647f5927d323572fdbdce`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:07:42.744143 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3cceb5d5-063f-4cb0-8cb5-c30b13dee367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:07:42.752646 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 697/1411] 10.222.0.50 () {58 vars in 1015 bytes} [Fri May 27 15:07:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:07:42.768529 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c6574f-ea45-42f9-924f-1e46bf51acbb None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:42.768820 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c6574f-ea45-42f9-924f-1e46bf51acbb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:42.769098 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c6574f-ea45-42f9-924f-1e46bf51acbb None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:42.770476 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 715/1412] 10.222.0.50 () {54 vars in 918 bytes} [Fri May 27 15:07:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:07:42.775550 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6b030-5ad3-4a8b-8083-7129aca51f50 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:42.775894 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6b030-5ad3-4a8b-8083-7129aca51f50 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:42.776215 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6b030-5ad3-4a8b-8083-7129aca51f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:42.794697 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8ff6b030-5ad3-4a8b-8083-7129aca51f50 None None] MFA Rules not processed for user `451a9232f95e494683d81b6078a318d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:07:42.795911 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8ff6b030-5ad3-4a8b-8083-7129aca51f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:07:42.805334 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 698/1413] 10.222.0.50 () {58 vars in 1015 bytes} [Fri May 27 15:07:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:07:45.499139 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaeea33-6ccd-40dc-9342-b001b1eb7a37 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:45.499695 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaeea33-6ccd-40dc-9342-b001b1eb7a37 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:45.499695 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaeea33-6ccd-40dc-9342-b001b1eb7a37 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:45.501015 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 716/1414] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 15:07:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:07:45.507322 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3e4ea-cbb8-4892-a484-534d351f2a4e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:45.507816 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3e4ea-cbb8-4892-a484-534d351f2a4e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:45.507927 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b3e4ea-cbb8-4892-a484-534d351f2a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:45.525804 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-17b3e4ea-cbb8-4892-a484-534d351f2a4e None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:07:45.527020 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-17b3e4ea-cbb8-4892-a484-534d351f2a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:07:45.534715 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 699/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 15:07:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:07:45.577345 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaeea33-6ccd-40dc-9342-b001b1eb7a37 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:07:45.592603 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaeea33-6ccd-40dc-9342-b001b1eb7a37 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:07:45.597924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6aa256-bbab-4303-adf8-d5ebe4e88972 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '451a9232f95e494683d81b6078a318d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:07:45.598555 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6aa256-bbab-4303-adf8-d5ebe4e88972 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:45.598840 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6aa256-bbab-4303-adf8-d5ebe4e88972 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:45.599142 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6aa256-bbab-4303-adf8-d5ebe4e88972 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:45.607831 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6aa256-bbab-4303-adf8-d5ebe4e88972 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:07:45.609402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6aa256-bbab-4303-adf8-d5ebe4e88972 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:07:45.618910 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 717/1416] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:07:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:07:46.597472 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffefe8f-58c2-4549-9cb7-c96a9dfbd4ca None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:46.598040 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffefe8f-58c2-4549-9cb7-c96a9dfbd4ca None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:46.598040 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffefe8f-58c2-4549-9cb7-c96a9dfbd4ca None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:46.599447 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 700/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 15:07:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:07:46.604631 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c403d03-7bac-49c3-9382-a748234c7bc1 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:46.605022 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c403d03-7bac-49c3-9382-a748234c7bc1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:46.605193 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c403d03-7bac-49c3-9382-a748234c7bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:46.623723 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2c403d03-7bac-49c3-9382-a748234c7bc1 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:07:46.624924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2c403d03-7bac-49c3-9382-a748234c7bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:07:46.633255 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 718/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 15:07:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:07:46.802011 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffefe8f-58c2-4549-9cb7-c96a9dfbd4ca None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:07:46.817043 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffefe8f-58c2-4549-9cb7-c96a9dfbd4ca None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:07:46.822648 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8274d442-5b26-4a83-8c64-51effca2b80c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '451a9232f95e494683d81b6078a318d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:07:46.823321 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274d442-5b26-4a83-8c64-51effca2b80c service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:46.823610 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274d442-5b26-4a83-8c64-51effca2b80c service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:46.823892 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274d442-5b26-4a83-8c64-51effca2b80c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:46.832058 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8274d442-5b26-4a83-8c64-51effca2b80c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:07:46.833641 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8274d442-5b26-4a83-8c64-51effca2b80c service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:07:46.843358 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 701/1419] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:07:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:07:47.778648 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf5a1b-4b88-43fc-a745-7926b12a00a0 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:47.779221 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf5a1b-4b88-43fc-a745-7926b12a00a0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:47.779221 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcf5a1b-4b88-43fc-a745-7926b12a00a0 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:47.780652 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 719/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 15:07:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:07:47.785705 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a5a99-d1dc-47f9-be49-1c6bed5793b9 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:47.786131 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a5a99-d1dc-47f9-be49-1c6bed5793b9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:47.786264 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a5a99-d1dc-47f9-be49-1c6bed5793b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:47.805283 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d82a5a99-d1dc-47f9-be49-1c6bed5793b9 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:07:47.806465 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d82a5a99-d1dc-47f9-be49-1c6bed5793b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:07:47.814168 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 702/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 15:07:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:07:47.967038 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf5a1b-4b88-43fc-a745-7926b12a00a0 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:07:47.981879 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcf5a1b-4b88-43fc-a745-7926b12a00a0 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:07:47.987285 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30d717f-6d0f-4b71-9eb0-d7df78943d39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '451a9232f95e494683d81b6078a318d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:07:47.987925 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d717f-6d0f-4b71-9eb0-d7df78943d39 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:47.988210 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d717f-6d0f-4b71-9eb0-d7df78943d39 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:47.988487 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30d717f-6d0f-4b71-9eb0-d7df78943d39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:47.996802 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30d717f-6d0f-4b71-9eb0-d7df78943d39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:07:47.998380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30d717f-6d0f-4b71-9eb0-d7df78943d39 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:07:48.007869 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 720/1422] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:07:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:07:48.918175 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c975e-4f59-4619-a11c-8c47b7edd444 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:48.918719 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c975e-4f59-4619-a11c-8c47b7edd444 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:48.918719 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c975e-4f59-4619-a11c-8c47b7edd444 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:48.921145 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 703/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 15:07:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:07:48.926225 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f5f5d-914d-4ead-8ef9-eb0fc627b175 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:48.926662 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f5f5d-914d-4ead-8ef9-eb0fc627b175 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:48.926782 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f5f5d-914d-4ead-8ef9-eb0fc627b175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:48.945427 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1c0f5f5d-914d-4ead-8ef9-eb0fc627b175 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:07:48.946612 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1c0f5f5d-914d-4ead-8ef9-eb0fc627b175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:07:48.954880 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 721/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 15:07:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:07:49.106828 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c975e-4f59-4619-a11c-8c47b7edd444 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:07:49.120965 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c975e-4f59-4619-a11c-8c47b7edd444 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:07:49.126436 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d692296-6703-4375-8718-91bd38549c3c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '451a9232f95e494683d81b6078a318d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:07:49.127121 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d692296-6703-4375-8718-91bd38549c3c service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:49.127409 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d692296-6703-4375-8718-91bd38549c3c service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:49.127687 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d692296-6703-4375-8718-91bd38549c3c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:49.136396 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d692296-6703-4375-8718-91bd38549c3c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:07:49.137980 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d692296-6703-4375-8718-91bd38549c3c service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:07:49.147865 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 704/1425] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:07:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:07:50.055766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ce1a4b-a5d5-482e-a7c7-45b7cbd6e812 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:50.056352 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ce1a4b-a5d5-482e-a7c7-45b7cbd6e812 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:50.056352 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ce1a4b-a5d5-482e-a7c7-45b7cbd6e812 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:50.057751 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 722/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 15:07:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:07:50.063104 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d964a-d52b-4535-998b-c4e9d39e31b3 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:50.063507 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d964a-d52b-4535-998b-c4e9d39e31b3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:50.063670 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76d964a-d52b-4535-998b-c4e9d39e31b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:50.082556 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c76d964a-d52b-4535-998b-c4e9d39e31b3 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:07:50.083822 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c76d964a-d52b-4535-998b-c4e9d39e31b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:07:50.091957 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 705/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 15:07:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:07:50.244996 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ce1a4b-a5d5-482e-a7c7-45b7cbd6e812 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:07:50.260318 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ce1a4b-a5d5-482e-a7c7-45b7cbd6e812 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:07:50.265704 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a145c-e8a8-4c6c-bb2e-f5cf59576607 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '451a9232f95e494683d81b6078a318d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:07:50.266337 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a145c-e8a8-4c6c-bb2e-f5cf59576607 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:50.266618 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a145c-e8a8-4c6c-bb2e-f5cf59576607 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:50.266923 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a145c-e8a8-4c6c-bb2e-f5cf59576607 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:50.275500 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054a145c-e8a8-4c6c-bb2e-f5cf59576607 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:07:50.277074 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054a145c-e8a8-4c6c-bb2e-f5cf59576607 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:07:50.286262 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 723/1428] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:07:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:07:51.147235 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d391b4f-dd95-4f8c-aef8-c571f39bf64b None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:51.147789 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d391b4f-dd95-4f8c-aef8-c571f39bf64b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:51.147789 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d391b4f-dd95-4f8c-aef8-c571f39bf64b None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:51.149205 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 706/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 15:07:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:07:51.154471 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288510e4-f21a-49a5-b6e8-7da323158146 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:51.154926 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288510e4-f21a-49a5-b6e8-7da323158146 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:51.155067 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288510e4-f21a-49a5-b6e8-7da323158146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:51.173591 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-288510e4-f21a-49a5-b6e8-7da323158146 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:07:51.174786 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-288510e4-f21a-49a5-b6e8-7da323158146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:07:51.183426 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 724/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 15:07:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:07:51.333834 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d391b4f-dd95-4f8c-aef8-c571f39bf64b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:07:51.348821 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d391b4f-dd95-4f8c-aef8-c571f39bf64b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:07:51.354899 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab221d3b-d73b-416a-ac59-14569c2a07b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '451a9232f95e494683d81b6078a318d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:07:51.355544 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab221d3b-d73b-416a-ac59-14569c2a07b8 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:51.355828 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab221d3b-d73b-416a-ac59-14569c2a07b8 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:51.356106 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab221d3b-d73b-416a-ac59-14569c2a07b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:51.364439 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab221d3b-d73b-416a-ac59-14569c2a07b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:07:51.366658 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab221d3b-d73b-416a-ac59-14569c2a07b8 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:07:51.376063 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 707/1431] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:07:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:07:52.238773 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7359d8e5-e928-4646-84cd-e0e2ef3214ed None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:52.239368 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7359d8e5-e928-4646-84cd-e0e2ef3214ed None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:52.239368 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7359d8e5-e928-4646-84cd-e0e2ef3214ed None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:52.241609 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 725/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 15:07:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:07:52.247016 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c154322-d754-4acc-8849-a69e340e0178 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:52.247607 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c154322-d754-4acc-8849-a69e340e0178 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:52.247607 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c154322-d754-4acc-8849-a69e340e0178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:52.266229 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-1c154322-d754-4acc-8849-a69e340e0178 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:07:52.267487 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1c154322-d754-4acc-8849-a69e340e0178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:07:52.276069 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 708/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 15:07:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:07:52.429926 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7359d8e5-e928-4646-84cd-e0e2ef3214ed None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:07:52.444926 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7359d8e5-e928-4646-84cd-e0e2ef3214ed None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:07:52.450372 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3afb4c-73f9-4bd8-bc7e-99e34c822aad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '451a9232f95e494683d81b6078a318d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:07:52.451046 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3afb4c-73f9-4bd8-bc7e-99e34c822aad service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:52.451330 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3afb4c-73f9-4bd8-bc7e-99e34c822aad service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:52.451603 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3afb4c-73f9-4bd8-bc7e-99e34c822aad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:52.460423 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3afb4c-73f9-4bd8-bc7e-99e34c822aad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:07:52.462030 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3afb4c-73f9-4bd8-bc7e-99e34c822aad service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:07:52.473080 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 726/1434] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:07:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:07:53.335651 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83fde70-3b02-4709-b7be-55a9771763d2 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:53.336196 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83fde70-3b02-4709-b7be-55a9771763d2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:53.336196 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83fde70-3b02-4709-b7be-55a9771763d2 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:53.337638 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 709/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 15:07:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 15:07:53.343257 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ffaf9d-ab27-42d5-9292-ac2bc34e5f2b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:53.343686 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ffaf9d-ab27-42d5-9292-ac2bc34e5f2b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:53.343822 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ffaf9d-ab27-42d5-9292-ac2bc34e5f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:53.361692 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-58ffaf9d-ab27-42d5-9292-ac2bc34e5f2b None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:07:53.362917 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-58ffaf9d-ab27-42d5-9292-ac2bc34e5f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:07:53.371301 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 727/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 15:07:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:07:53.527212 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83fde70-3b02-4709-b7be-55a9771763d2 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:07:53.542042 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83fde70-3b02-4709-b7be-55a9771763d2 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:07:53.547544 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8258b61-e587-4a9f-a269-094396bd7360 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '451a9232f95e494683d81b6078a318d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:07:53.548201 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8258b61-e587-4a9f-a269-094396bd7360 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:53.548485 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8258b61-e587-4a9f-a269-094396bd7360 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:53.548764 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8258b61-e587-4a9f-a269-094396bd7360 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:53.557983 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8258b61-e587-4a9f-a269-094396bd7360 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:07:53.559627 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8258b61-e587-4a9f-a269-094396bd7360 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:07:53.569108 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 710/1437] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:07:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:07:54.441418 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a9602d-11c6-4a82-9f41-8277af4d305b None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:54.441975 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a9602d-11c6-4a82-9f41-8277af4d305b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:54.441975 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a9602d-11c6-4a82-9f41-8277af4d305b None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:54.443305 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 728/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 15:07:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:07:54.449853 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8633151-2b69-45c3-80e0-694eca106b7d None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:54.450576 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8633151-2b69-45c3-80e0-694eca106b7d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:54.450576 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8633151-2b69-45c3-80e0-694eca106b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:54.469287 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a8633151-2b69-45c3-80e0-694eca106b7d None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:07:54.470516 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a8633151-2b69-45c3-80e0-694eca106b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:07:54.478628 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 711/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 15:07:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:07:54.628625 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a9602d-11c6-4a82-9f41-8277af4d305b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:07:54.643250 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a9602d-11c6-4a82-9f41-8277af4d305b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:07:54.648600 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa57d85-deb0-45dc-9a70-2722fe4606d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '451a9232f95e494683d81b6078a318d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:07:54.649245 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa57d85-deb0-45dc-9a70-2722fe4606d3 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:54.649557 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa57d85-deb0-45dc-9a70-2722fe4606d3 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:54.649837 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa57d85-deb0-45dc-9a70-2722fe4606d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:54.658156 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa57d85-deb0-45dc-9a70-2722fe4606d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:07:54.659788 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa57d85-deb0-45dc-9a70-2722fe4606d3 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:07:54.669809 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 729/1440] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:07:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:07:56.810416 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e397e36-23fb-41c5-89fd-6d8c8f175645 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:56.810969 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e397e36-23fb-41c5-89fd-6d8c8f175645 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:56.811108 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e397e36-23fb-41c5-89fd-6d8c8f175645 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:56.812401 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 712/1441] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 15:07:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:07:56.817981 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8500cc7c-14ca-46cd-ba8b-ca350676aee6 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:56.818415 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8500cc7c-14ca-46cd-ba8b-ca350676aee6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:56.818530 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8500cc7c-14ca-46cd-ba8b-ca350676aee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:56.837629 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8500cc7c-14ca-46cd-ba8b-ca350676aee6 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:07:56.838876 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8500cc7c-14ca-46cd-ba8b-ca350676aee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:07:56.847189 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 730/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 15:07:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:07:56.885843 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e397e36-23fb-41c5-89fd-6d8c8f175645 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:07:56.887751 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7e397e36-23fb-41c5-89fd-6d8c8f175645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:07:56.903756 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e397e36-23fb-41c5-89fd-6d8c8f175645 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:07:56.907515 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af0f511-f00f-41b0-8d24-7345548744ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:07:56.908163 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af0f511-f00f-41b0-8d24-7345548744ce service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:56.908448 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af0f511-f00f-41b0-8d24-7345548744ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:56.908727 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af0f511-f00f-41b0-8d24-7345548744ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:56.920162 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af0f511-f00f-41b0-8d24-7345548744ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:07:56.921791 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af0f511-f00f-41b0-8d24-7345548744ce service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:07:56.932711 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 713/1443] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:07:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:07:59.387271 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb83cf6-a5e8-4eb3-9afe-024edeecd8d8 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:59.387787 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb83cf6-a5e8-4eb3-9afe-024edeecd8d8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:59.387787 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb83cf6-a5e8-4eb3-9afe-024edeecd8d8 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:59.389139 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 731/1444] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 15:07:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:07:59.394548 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db2f4f-2457-4e04-8fe7-45b9e2be485f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:59.394990 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db2f4f-2457-4e04-8fe7-45b9e2be485f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:59.395145 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db2f4f-2457-4e04-8fe7-45b9e2be485f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:59.413405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f1db2f4f-2457-4e04-8fe7-45b9e2be485f None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:07:59.414595 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f1db2f4f-2457-4e04-8fe7-45b9e2be485f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:07:59.422508 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 714/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 15:07:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:07:59.459894 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb83cf6-a5e8-4eb3-9afe-024edeecd8d8 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:07:59.471833 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb83cf6-a5e8-4eb3-9afe-024edeecd8d8 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:07:59.475301 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299ea43c-704b-40ee-a060-d6c2c6d9004a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:07:59.475967 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ea43c-704b-40ee-a060-d6c2c6d9004a service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:07:59.476247 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ea43c-704b-40ee-a060-d6c2c6d9004a service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:07:59.476520 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299ea43c-704b-40ee-a060-d6c2c6d9004a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:07:59.486667 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299ea43c-704b-40ee-a060-d6c2c6d9004a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:07:59.488138 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299ea43c-704b-40ee-a060-d6c2c6d9004a service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:07:59.497864 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 732/1446] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:07:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:08:02.156623 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba652a-4263-4acb-b9db-a6d3e3de7f27 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:02.157062 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba652a-4263-4acb-b9db-a6d3e3de7f27 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:02.157200 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba652a-4263-4acb-b9db-a6d3e3de7f27 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:02.158866 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 715/1447] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 15:08:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:08:02.165384 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f66de6-c537-45f9-89f3-380ec056df25 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:02.165830 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f66de6-c537-45f9-89f3-380ec056df25 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:02.166014 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f66de6-c537-45f9-89f3-380ec056df25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:02.186633 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-65f66de6-c537-45f9-89f3-380ec056df25 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:08:02.188033 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-65f66de6-c537-45f9-89f3-380ec056df25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:08:02.198457 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 733/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 15:08:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:08:02.239605 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ba652a-4263-4acb-b9db-a6d3e3de7f27 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:08:02.251373 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ba652a-4263-4acb-b9db-a6d3e3de7f27 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:08:02.255111 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fa5772-0f5e-498f-9c17-c86b8dfae6b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:08:02.255766 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa5772-0f5e-498f-9c17-c86b8dfae6b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:02.256033 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa5772-0f5e-498f-9c17-c86b8dfae6b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:02.256312 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fa5772-0f5e-498f-9c17-c86b8dfae6b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:02.266139 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fa5772-0f5e-498f-9c17-c86b8dfae6b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:08:02.267602 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fa5772-0f5e-498f-9c17-c86b8dfae6b5 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:08:02.277144 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 716/1449] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:08:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:08:04.625109 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d628f2-9084-4b57-84b8-ab5563147e19 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:04.625719 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d628f2-9084-4b57-84b8-ab5563147e19 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:04.625719 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d628f2-9084-4b57-84b8-ab5563147e19 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:04.627080 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 734/1450] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 15:08:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:08:04.632296 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf70ce47-0186-4b93-a83a-3257006292ea None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:04.632709 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf70ce47-0186-4b93-a83a-3257006292ea None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:04.632852 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf70ce47-0186-4b93-a83a-3257006292ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:04.652121 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-bf70ce47-0186-4b93-a83a-3257006292ea None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:08:04.653324 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bf70ce47-0186-4b93-a83a-3257006292ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:08:04.661052 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 717/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 15:08:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:08:04.701803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d628f2-9084-4b57-84b8-ab5563147e19 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:08:04.713619 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d628f2-9084-4b57-84b8-ab5563147e19 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:08:04.717397 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655a8285-382c-407b-8540-3a3aa2c0bc09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:08:04.718031 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655a8285-382c-407b-8540-3a3aa2c0bc09 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:04.718313 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655a8285-382c-407b-8540-3a3aa2c0bc09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:04.718588 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655a8285-382c-407b-8540-3a3aa2c0bc09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:04.728479 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655a8285-382c-407b-8540-3a3aa2c0bc09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:08:04.729963 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655a8285-382c-407b-8540-3a3aa2c0bc09 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:08:04.739755 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 735/1452] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:08:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:08:07.048910 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edede077-31ca-4592-876b-18d90e9404d8 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:07.049564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edede077-31ca-4592-876b-18d90e9404d8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:07.049564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edede077-31ca-4592-876b-18d90e9404d8 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:07.050883 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 718/1453] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 15:08:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 15:08:07.059829 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dca15a-213d-4dff-a9ff-fa5668e9a309 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:07.060343 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dca15a-213d-4dff-a9ff-fa5668e9a309 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:07.060446 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dca15a-213d-4dff-a9ff-fa5668e9a309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:07.080348 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a2dca15a-213d-4dff-a9ff-fa5668e9a309 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:08:07.081539 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a2dca15a-213d-4dff-a9ff-fa5668e9a309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:08:07.090932 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 736/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 15:08:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:08:07.129301 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edede077-31ca-4592-876b-18d90e9404d8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:08:07.140707 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edede077-31ca-4592-876b-18d90e9404d8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:08:07.144262 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25de882-fb13-482d-b498-6ecb4bca9efd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:08:07.144905 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25de882-fb13-482d-b498-6ecb4bca9efd service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:07.145200 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25de882-fb13-482d-b498-6ecb4bca9efd service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:07.145477 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25de882-fb13-482d-b498-6ecb4bca9efd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:07.156280 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25de882-fb13-482d-b498-6ecb4bca9efd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:08:07.157504 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25de882-fb13-482d-b498-6ecb4bca9efd service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:08:07.171905 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 719/1455] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:08:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:08:09.595087 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d6e37-46c2-4a39-92ca-7d62f45b5d52 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:09.595688 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d6e37-46c2-4a39-92ca-7d62f45b5d52 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:09.595688 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d6e37-46c2-4a39-92ca-7d62f45b5d52 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:09.597198 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 737/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 15:08:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:08:09.606118 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7772268-b5e3-481c-b07c-23cc2be3dc32 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:09.606620 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7772268-b5e3-481c-b07c-23cc2be3dc32 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:09.606756 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7772268-b5e3-481c-b07c-23cc2be3dc32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:09.629958 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f7772268-b5e3-481c-b07c-23cc2be3dc32 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:08:09.631220 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f7772268-b5e3-481c-b07c-23cc2be3dc32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:08:09.640115 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 720/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 15:08:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:08:09.683909 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034d6e37-46c2-4a39-92ca-7d62f45b5d52 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:08:09.695932 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034d6e37-46c2-4a39-92ca-7d62f45b5d52 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:08:09.700210 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb4d589-4012-44b3-9fde-205542e34de2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:08:09.701026 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4d589-4012-44b3-9fde-205542e34de2 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:09.701426 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4d589-4012-44b3-9fde-205542e34de2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:09.701773 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4d589-4012-44b3-9fde-205542e34de2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:09.712763 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb4d589-4012-44b3-9fde-205542e34de2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:08:09.714238 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb4d589-4012-44b3-9fde-205542e34de2 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:08:09.730334 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 738/1458] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:08:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:08:12.037899 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d36d795-2a0f-4bd8-834f-08218e270652 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:12.038468 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d36d795-2a0f-4bd8-834f-08218e270652 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:12.038468 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d36d795-2a0f-4bd8-834f-08218e270652 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:12.040677 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 721/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 15:08:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:08:12.046259 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93777255-af45-4a89-909c-351aee0cfca3 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:12.046715 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93777255-af45-4a89-909c-351aee0cfca3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:12.046810 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93777255-af45-4a89-909c-351aee0cfca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:12.052949 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=70 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:08:12.067451 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-93777255-af45-4a89-909c-351aee0cfca3 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:08:12.068683 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-93777255-af45-4a89-909c-351aee0cfca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:08:12.077530 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 739/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 15:08:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:08:12.115615 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d36d795-2a0f-4bd8-834f-08218e270652 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:08:12.121170 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=73 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:08:12.127719 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d36d795-2a0f-4bd8-834f-08218e270652 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:08:12.131272 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29690d-81bf-4a8f-b907-c4f3021812ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:08:12.131912 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29690d-81bf-4a8f-b907-c4f3021812ba service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:12.132194 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29690d-81bf-4a8f-b907-c4f3021812ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:12.132467 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29690d-81bf-4a8f-b907-c4f3021812ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:12.142427 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a29690d-81bf-4a8f-b907-c4f3021812ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:08:12.143903 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a29690d-81bf-4a8f-b907-c4f3021812ba service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:08:12.154113 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 722/1461] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:08:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:08:14.513376 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b7a220-9ef3-40db-a04f-2068372b8eee None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:14.513928 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b7a220-9ef3-40db-a04f-2068372b8eee None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:14.513928 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b7a220-9ef3-40db-a04f-2068372b8eee None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:14.515303 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 740/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 15:08:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:08:14.520780 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654a7cf-8d5b-4af8-aac7-9871820f3a2b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:14.521243 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654a7cf-8d5b-4af8-aac7-9871820f3a2b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:14.521355 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8654a7cf-8d5b-4af8-aac7-9871820f3a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:14.539582 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8654a7cf-8d5b-4af8-aac7-9871820f3a2b None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:08:14.540784 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8654a7cf-8d5b-4af8-aac7-9871820f3a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:08:14.548702 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 723/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 15:08:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:08:14.589764 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b7a220-9ef3-40db-a04f-2068372b8eee None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:08:14.601288 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b7a220-9ef3-40db-a04f-2068372b8eee None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:08:14.604851 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c30c00-b814-492c-a431-4669f27e59ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:08:14.605501 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c30c00-b814-492c-a431-4669f27e59ee service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:14.605812 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c30c00-b814-492c-a431-4669f27e59ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:14.606092 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c30c00-b814-492c-a431-4669f27e59ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:14.616197 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c30c00-b814-492c-a431-4669f27e59ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:08:14.617812 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c30c00-b814-492c-a431-4669f27e59ee service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:08:14.627566 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 741/1464] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:08:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:08:16.953591 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a7ba3-969e-4968-8309-0e16572aff8f None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:16.954139 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a7ba3-969e-4968-8309-0e16572aff8f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:16.954139 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31a7ba3-969e-4968-8309-0e16572aff8f None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:16.955522 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 724/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 15:08:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:08:16.961105 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef134a90-cae6-44c3-8d7e-87d1beac53c7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:16.962282 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef134a90-cae6-44c3-8d7e-87d1beac53c7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:16.962282 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef134a90-cae6-44c3-8d7e-87d1beac53c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:16.986948 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-ef134a90-cae6-44c3-8d7e-87d1beac53c7 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:08:16.988184 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ef134a90-cae6-44c3-8d7e-87d1beac53c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:08:16.996632 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 742/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 15:08:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:08:17.033213 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a7ba3-969e-4968-8309-0e16572aff8f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:08:17.045045 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31a7ba3-969e-4968-8309-0e16572aff8f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:08:17.048633 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01539aa-467c-46a4-994c-fba6db219e7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:08:17.049267 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01539aa-467c-46a4-994c-fba6db219e7e service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:17.049552 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01539aa-467c-46a4-994c-fba6db219e7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:17.049833 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01539aa-467c-46a4-994c-fba6db219e7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:17.058882 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01539aa-467c-46a4-994c-fba6db219e7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:08:17.060323 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01539aa-467c-46a4-994c-fba6db219e7e service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:08:17.069673 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 725/1467] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:08:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:08:19.357528 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957e617-07f4-4d30-8694-4563a0421351 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:19.358082 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957e617-07f4-4d30-8694-4563a0421351 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:19.358082 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957e617-07f4-4d30-8694-4563a0421351 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:19.359500 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 743/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 15:08:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:08:19.364767 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e22d7-392b-4ed7-9893-b78f81b2724b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:19.365211 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e22d7-392b-4ed7-9893-b78f81b2724b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:19.365360 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e22d7-392b-4ed7-9893-b78f81b2724b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:19.384697 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-096e22d7-392b-4ed7-9893-b78f81b2724b None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:08:19.385885 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-096e22d7-392b-4ed7-9893-b78f81b2724b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:08:19.393633 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 726/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 15:08:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:08:19.429464 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b957e617-07f4-4d30-8694-4563a0421351 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:08:19.440783 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b957e617-07f4-4d30-8694-4563a0421351 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:08:19.444299 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e532c1cc-f43f-4910-9585-b40ae6b12156 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:08:19.444930 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532c1cc-f43f-4910-9585-b40ae6b12156 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:19.445214 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532c1cc-f43f-4910-9585-b40ae6b12156 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:19.445571 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532c1cc-f43f-4910-9585-b40ae6b12156 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:19.457235 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e532c1cc-f43f-4910-9585-b40ae6b12156 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:08:19.458776 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e532c1cc-f43f-4910-9585-b40ae6b12156 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:08:19.469658 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 744/1470] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:08:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:08:29.378722 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=17 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:08:29.464522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=17 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:08:53.706597 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4cce9-3ace-40d7-9491-782bbfe65c6e None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:53.707480 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4cce9-3ace-40d7-9491-782bbfe65c6e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:53.707658 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4cce9-3ace-40d7-9491-782bbfe65c6e None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:53.709532 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 727/1471] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:08:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:08:53.716053 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb9093-0a2c-4140-b6ca-f95760b9d2be None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:53.716543 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb9093-0a2c-4140-b6ca-f95760b9d2be None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:53.716642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deb9093-0a2c-4140-b6ca-f95760b9d2be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:53.734866 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0deb9093-0a2c-4140-b6ca-f95760b9d2be None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:08:53.736077 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0deb9093-0a2c-4140-b6ca-f95760b9d2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:08:53.744159 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 745/1472] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:08:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:08:53.792387 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34fb4a1-b3bd-49d2-a367-ff154876724b None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:53.792679 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34fb4a1-b3bd-49d2-a367-ff154876724b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:53.792963 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34fb4a1-b3bd-49d2-a367-ff154876724b None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:53.794355 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 728/1473] 10.222.0.50 () {54 vars in 962 bytes} [Fri May 27 15:08:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:08:53.799217 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61315d86-e2af-4365-be44-989a0b83223a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:53.799579 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61315d86-e2af-4365-be44-989a0b83223a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:53.799860 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61315d86-e2af-4365-be44-989a0b83223a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:53.818275 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-61315d86-e2af-4365-be44-989a0b83223a None None] MFA Rules not processed for user `1afda9d6b1d64e6ab400b26124cf1290`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:08:53.819502 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-61315d86-e2af-4365-be44-989a0b83223a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:08:53.826222 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 746/1474] 10.222.0.50 () {58 vars in 1059 bytes} [Fri May 27 15:08:53 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:08:53.833633 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34fb4a1-b3bd-49d2-a367-ff154876724b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:08:53.845172 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34fb4a1-b3bd-49d2-a367-ff154876724b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:08:53.848629 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263b9417-5007-4287-86ef-4753a93fb295 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1afda9d6b1d64e6ab400b26124cf1290', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:08:53.849254 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b9417-5007-4287-86ef-4753a93fb295 service glance] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:53.849534 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b9417-5007-4287-86ef-4753a93fb295 service glance] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:53.849809 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263b9417-5007-4287-86ef-4753a93fb295 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:53.859515 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263b9417-5007-4287-86ef-4753a93fb295 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:08:53.860930 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263b9417-5007-4287-86ef-4753a93fb295 service glance] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:08:53.870825 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 729/1475] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:08:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:08:56.252877 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aeb82e-331b-4cb1-9973-20d05fd2d6bc None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:56.253467 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aeb82e-331b-4cb1-9973-20d05fd2d6bc None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:56.253467 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aeb82e-331b-4cb1-9973-20d05fd2d6bc None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:56.254831 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 747/1476] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:08:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:08:56.261728 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504ea54-d619-409a-9bec-ba5cb2840eb5 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:56.262261 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504ea54-d619-409a-9bec-ba5cb2840eb5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:56.262360 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c504ea54-d619-409a-9bec-ba5cb2840eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:56.282413 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c504ea54-d619-409a-9bec-ba5cb2840eb5 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:08:56.283656 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c504ea54-d619-409a-9bec-ba5cb2840eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:08:56.292212 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 730/1477] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:08:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:08:56.317413 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aeb82e-331b-4cb1-9973-20d05fd2d6bc None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:08:56.329912 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aeb82e-331b-4cb1-9973-20d05fd2d6bc None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:08:56.334317 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94439b-d111-4f40-9f58-a0f47c5c6cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:08:56.334999 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94439b-d111-4f40-9f58-a0f47c5c6cf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:56.335285 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94439b-d111-4f40-9f58-a0f47c5c6cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:56.335576 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94439b-d111-4f40-9f58-a0f47c5c6cf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:56.336317 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c94439b-d111-4f40-9f58-a0f47c5c6cf9 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:08:56.338084 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: WARNING py.warnings [None req-6c94439b-d111-4f40-9f58-a0f47c5c6cf9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 15:08:56.338084 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: warnings.warn(msg) May 27 15:08:56.338084 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: May 27 15:08:56.339276 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c94439b-d111-4f40-9f58-a0f47c5c6cf9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:08:56.359712 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 748/1478] 10.222.0.50 () {56 vars in 1108 bytes} [Fri May 27 15:08:56 2022] GET /identity/v3/projects => generated 2553 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:08:58.669191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b30de-99b3-4a7b-a9fd-345024384ebe None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:58.669728 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b30de-99b3-4a7b-a9fd-345024384ebe None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:58.669728 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56b30de-99b3-4a7b-a9fd-345024384ebe None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:58.671093 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 731/1479] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:08:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:08:58.676529 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388e804-34a0-4035-ac98-82a09d08787e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:58.676973 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388e804-34a0-4035-ac98-82a09d08787e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:58.677096 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388e804-34a0-4035-ac98-82a09d08787e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:58.696140 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f388e804-34a0-4035-ac98-82a09d08787e None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:08:58.697272 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f388e804-34a0-4035-ac98-82a09d08787e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:08:58.705564 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 749/1480] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:08:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:08:58.750318 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b30de-99b3-4a7b-a9fd-345024384ebe None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:08:58.770216 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56b30de-99b3-4a7b-a9fd-345024384ebe None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:08:58.775735 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1d03c0-7230-46c2-bcb6-57caba67f536 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '451a9232f95e494683d81b6078a318d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:08:58.776372 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d03c0-7230-46c2-bcb6-57caba67f536 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:08:58.776653 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d03c0-7230-46c2-bcb6-57caba67f536 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:08:58.776924 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d03c0-7230-46c2-bcb6-57caba67f536 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:08:58.785299 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1d03c0-7230-46c2-bcb6-57caba67f536 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:08:58.786991 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1d03c0-7230-46c2-bcb6-57caba67f536 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:08:58.798162 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 732/1481] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:08:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:09:01.254801 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5001a2-9f7e-4493-967f-290906980175 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:09:01.255388 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5001a2-9f7e-4493-967f-290906980175 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:09:01.255388 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5001a2-9f7e-4493-967f-290906980175 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:09:01.256740 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 750/1482] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:09:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:09:01.262167 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4d89c-bf90-468e-b8b8-67e226d5e8d6 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:09:01.262625 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4d89c-bf90-468e-b8b8-67e226d5e8d6 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:09:01.262735 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4d89c-bf90-468e-b8b8-67e226d5e8d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:09:01.280735 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7ca4d89c-bf90-468e-b8b8-67e226d5e8d6 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:09:01.281908 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7ca4d89c-bf90-468e-b8b8-67e226d5e8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:09:01.290424 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 733/1483] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:09:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:09:01.325681 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5001a2-9f7e-4493-967f-290906980175 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:09:01.338530 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5001a2-9f7e-4493-967f-290906980175 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:09:01.342371 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4435d1-5a30-405b-a1a3-bdc814c15c33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:09:01.343041 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4435d1-5a30-405b-a1a3-bdc814c15c33 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:09:01.343332 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4435d1-5a30-405b-a1a3-bdc814c15c33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:09:01.343613 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4435d1-5a30-405b-a1a3-bdc814c15c33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:09:01.353265 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4435d1-5a30-405b-a1a3-bdc814c15c33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:09:01.354716 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4435d1-5a30-405b-a1a3-bdc814c15c33 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:09:01.364324 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 751/1484] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:09:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:09:03.656453 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74547a4d-17db-40ea-9da8-415be6701a9d None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:09:03.657027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74547a4d-17db-40ea-9da8-415be6701a9d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:09:03.657027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74547a4d-17db-40ea-9da8-415be6701a9d None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:09:03.658366 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 734/1485] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:09:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:09:03.664421 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0daa72-1c64-49cb-9c34-5c3c44792233 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:09:03.664914 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0daa72-1c64-49cb-9c34-5c3c44792233 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:09:03.665279 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0daa72-1c64-49cb-9c34-5c3c44792233 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:09:03.683416 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8a0daa72-1c64-49cb-9c34-5c3c44792233 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:09:03.684599 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8a0daa72-1c64-49cb-9c34-5c3c44792233 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:09:03.692608 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 752/1486] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:09:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:09:03.741772 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74547a4d-17db-40ea-9da8-415be6701a9d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:09:03.754110 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74547a4d-17db-40ea-9da8-415be6701a9d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:09:03.758318 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53254bf5-6b3e-4975-bb27-7bb123a4f44a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:09:03.759083 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53254bf5-6b3e-4975-bb27-7bb123a4f44a service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:09:03.759418 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53254bf5-6b3e-4975-bb27-7bb123a4f44a service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:09:03.759731 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53254bf5-6b3e-4975-bb27-7bb123a4f44a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:09:03.769391 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53254bf5-6b3e-4975-bb27-7bb123a4f44a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:09:03.770816 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53254bf5-6b3e-4975-bb27-7bb123a4f44a service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:09:03.781507 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 735/1487] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:09:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:09:06.209837 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c7275c-9b03-4b39-ad40-6320dc451244 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:09:06.210551 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c7275c-9b03-4b39-ad40-6320dc451244 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:09:06.211030 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c7275c-9b03-4b39-ad40-6320dc451244 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:09:06.212562 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 753/1488] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:09:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:09:06.218433 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1723e0-beef-40d8-a1a9-f30f5c649e6b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:09:06.218912 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1723e0-beef-40d8-a1a9-f30f5c649e6b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:09:06.219025 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1723e0-beef-40d8-a1a9-f30f5c649e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:09:06.238025 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5d1723e0-beef-40d8-a1a9-f30f5c649e6b None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:09:06.239265 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5d1723e0-beef-40d8-a1a9-f30f5c649e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:09:06.247941 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 736/1489] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:09:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:09:06.287501 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c7275c-9b03-4b39-ad40-6320dc451244 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:09:06.300219 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c7275c-9b03-4b39-ad40-6320dc451244 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:09:06.304657 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c47569d-1ad7-4397-9e5c-469d222fa8f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:09:06.305411 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47569d-1ad7-4397-9e5c-469d222fa8f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:09:06.305672 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47569d-1ad7-4397-9e5c-469d222fa8f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:09:06.306064 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47569d-1ad7-4397-9e5c-469d222fa8f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:09:06.315164 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c47569d-1ad7-4397-9e5c-469d222fa8f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:09:06.316578 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c47569d-1ad7-4397-9e5c-469d222fa8f0 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:09:06.326215 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 754/1490] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:09:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:09:08.831993 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926b535-f59c-45ff-8084-47ab46647e0a None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:09:08.832531 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926b535-f59c-45ff-8084-47ab46647e0a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:09:08.832531 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926b535-f59c-45ff-8084-47ab46647e0a None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:09:08.833918 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 737/1491] 10.222.0.50 () {54 vars in 926 bytes} [Fri May 27 15:09:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:09:08.840271 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51150983-6d1a-4fdd-8833-630e1d4ba8c2 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:09:08.840720 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51150983-6d1a-4fdd-8833-630e1d4ba8c2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:09:08.841068 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51150983-6d1a-4fdd-8833-630e1d4ba8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:09:08.859222 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-51150983-6d1a-4fdd-8833-630e1d4ba8c2 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:09:08.860416 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-51150983-6d1a-4fdd-8833-630e1d4ba8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:09:08.868167 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 755/1492] 10.222.0.50 () {58 vars in 1023 bytes} [Fri May 27 15:09:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:09:08.907248 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926b535-f59c-45ff-8084-47ab46647e0a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:09:08.918798 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926b535-f59c-45ff-8084-47ab46647e0a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:09:08.922356 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58799b35-8c4f-44d5-8d53-2c7eff0314a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:09:08.923036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58799b35-8c4f-44d5-8d53-2c7eff0314a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:09:08.923318 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58799b35-8c4f-44d5-8d53-2c7eff0314a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:09:08.923596 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58799b35-8c4f-44d5-8d53-2c7eff0314a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:09:08.932953 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58799b35-8c4f-44d5-8d53-2c7eff0314a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:09:08.934390 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58799b35-8c4f-44d5-8d53-2c7eff0314a1 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:09:08.943947 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 738/1493] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:09:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:09:18.853964 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=33 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:09:18.939700 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=33 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:10:50.773759 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c23478-9b5b-46ea-b533-b32e212178bb None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:10:50.774257 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c23478-9b5b-46ea-b533-b32e212178bb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:10:50.774454 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c23478-9b5b-46ea-b533-b32e212178bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:10:50.792360 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-76c23478-9b5b-46ea-b533-b32e212178bb None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:10:50.793523 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-76c23478-9b5b-46ea-b533-b32e212178bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:10:50.802196 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 756/1494] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:10:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:10:50.811479 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e715d1-29a2-4ef5-a39f-30cc6ddb7e78 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:10:50.811965 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e715d1-29a2-4ef5-a39f-30cc6ddb7e78 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:10:50.812071 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e715d1-29a2-4ef5-a39f-30cc6ddb7e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:10:50.831282 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a6e715d1-29a2-4ef5-a39f-30cc6ddb7e78 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:10:50.833064 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a6e715d1-29a2-4ef5-a39f-30cc6ddb7e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:10:50.840802 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 739/1495] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:10:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:10:50.845632 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c23478-9b5b-46ea-b533-b32e212178bb None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:10:50.856025 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c23478-9b5b-46ea-b533-b32e212178bb None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:10:50.860431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e97ae-06a2-4545-a8dc-c83eecc91cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:10:50.861068 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e97ae-06a2-4545-a8dc-c83eecc91cfe admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:10:50.861346 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e97ae-06a2-4545-a8dc-c83eecc91cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:10:50.861617 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e97ae-06a2-4545-a8dc-c83eecc91cfe admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:10:50.862325 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7e97ae-06a2-4545-a8dc-c83eecc91cfe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:10:50.862627 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7e97ae-06a2-4545-a8dc-c83eecc91cfe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:10:50.863871 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: WARNING py.warnings [None req-8f7e97ae-06a2-4545-a8dc-c83eecc91cfe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 27 15:10:50.863871 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: warnings.warn(msg) May 27 15:10:50.863871 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: May 27 15:10:50.864740 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7e97ae-06a2-4545-a8dc-c83eecc91cfe admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:10:50.876131 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 757/1496] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:10:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:10:50.884084 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e715d1-29a2-4ef5-a39f-30cc6ddb7e78 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:10:50.893124 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e715d1-29a2-4ef5-a39f-30cc6ddb7e78 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:10:50.898135 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7786fe9-961f-4c51-b0a5-f0c1e0723714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:10:50.898783 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7786fe9-961f-4c51-b0a5-f0c1e0723714 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:10:50.899106 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7786fe9-961f-4c51-b0a5-f0c1e0723714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:10:50.899385 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7786fe9-961f-4c51-b0a5-f0c1e0723714 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:10:50.900084 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7786fe9-961f-4c51-b0a5-f0c1e0723714 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:10:50.901865 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: WARNING py.warnings [None req-c7786fe9-961f-4c51-b0a5-f0c1e0723714 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 15:10:50.901865 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: warnings.warn(msg) May 27 15:10:50.901865 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: May 27 15:10:50.903123 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7786fe9-961f-4c51-b0a5-f0c1e0723714 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:10:50.938626 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 740/1497] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:10:50 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:10:50.942835 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e97ae-06a2-4545-a8dc-c83eecc91cfe admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:10:50.952826 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e97ae-06a2-4545-a8dc-c83eecc91cfe admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:10:51.007858 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13fb73e-ae50-4d96-8408-abc1dc4349e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:10:51.008442 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13fb73e-ae50-4d96-8408-abc1dc4349e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:10:51.008723 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13fb73e-ae50-4d96-8408-abc1dc4349e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:10:51.008998 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13fb73e-ae50-4d96-8408-abc1dc4349e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:10:51.009686 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13fb73e-ae50-4d96-8408-abc1dc4349e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:10:51.010686 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: WARNING py.warnings [None req-d13fb73e-ae50-4d96-8408-abc1dc4349e0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 15:10:51.010686 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: warnings.warn(msg) May 27 15:10:51.010686 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: May 27 15:10:51.011914 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13fb73e-ae50-4d96-8408-abc1dc4349e0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:10:51.031543 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 758/1498] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:10:50 2022] POST /identity/v3/users => generated 289 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:10:51.036612 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7786fe9-961f-4c51-b0a5-f0c1e0723714 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:10:51.046636 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7786fe9-961f-4c51-b0a5-f0c1e0723714 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:10:51.051709 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6200b86-6dcd-4527-bf51-2e4324050a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:10:51.052363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6200b86-6dcd-4527-bf51-2e4324050a7d admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:10:51.052650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6200b86-6dcd-4527-bf51-2e4324050a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:10:51.052935 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6200b86-6dcd-4527-bf51-2e4324050a7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:10:51.053654 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6200b86-6dcd-4527-bf51-2e4324050a7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:10:51.054933 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: WARNING py.warnings [None req-d6200b86-6dcd-4527-bf51-2e4324050a7d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 27 15:10:51.054933 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: warnings.warn(msg) May 27 15:10:51.054933 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: May 27 15:10:51.055976 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6200b86-6dcd-4527-bf51-2e4324050a7d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:10:51.069387 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 741/1499] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:10:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:10:51.073673 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13fb73e-ae50-4d96-8408-abc1dc4349e0 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:10:51.083975 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13fb73e-ae50-4d96-8408-abc1dc4349e0 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:10:51.088502 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96434d5-c271-4814-93d6-bdf176348b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:10:51.089150 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96434d5-c271-4814-93d6-bdf176348b6f admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:10:51.089431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96434d5-c271-4814-93d6-bdf176348b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:10:51.089705 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96434d5-c271-4814-93d6-bdf176348b6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:10:51.090421 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96434d5-c271-4814-93d6-bdf176348b6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:10:51.091452 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: WARNING py.warnings [None req-d96434d5-c271-4814-93d6-bdf176348b6f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 27 15:10:51.091452 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: warnings.warn(msg) May 27 15:10:51.091452 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: May 27 15:10:51.092594 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96434d5-c271-4814-93d6-bdf176348b6f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:10:51.105899 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 759/1500] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:10:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:10:51.109993 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6200b86-6dcd-4527-bf51-2e4324050a7d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:10:51.119680 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6200b86-6dcd-4527-bf51-2e4324050a7d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:10:51.124419 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901d0407-15a6-40ed-bce7-a812ad6e543f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:10:51.125294 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d0407-15a6-40ed-bce7-a812ad6e543f admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:10:51.125577 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d0407-15a6-40ed-bce7-a812ad6e543f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:10:51.125854 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901d0407-15a6-40ed-bce7-a812ad6e543f admin admin] PATH_INFO: `/v3/projects/ca13243dae1546d49c6b2bce160850fb/users/08bc0ca115c8459792f49e96eb67f5d5/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:10:51.141367 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901d0407-15a6-40ed-bce7-a812ad6e543f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca13243dae1546d49c6b2bce160850fb, user_id=08bc0ca115c8459792f49e96eb67f5d5, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:10:51.142403 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: WARNING py.warnings [None req-901d0407-15a6-40ed-bce7-a812ad6e543f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 15:10:51.142403 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: warnings.warn(msg) May 27 15:10:51.142403 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: May 27 15:10:51.144438 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901d0407-15a6-40ed-bce7-a812ad6e543f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:10:51.161143 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 742/1501] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:10:51 2022] PUT /identity/v3/projects/ca13243dae1546d49c6b2bce160850fb/users/08bc0ca115c8459792f49e96eb67f5d5/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:10:51.172426 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b4acc3-8d21-4d60-9e32-f0cb1c2ed779 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:10:51.172722 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b4acc3-8d21-4d60-9e32-f0cb1c2ed779 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:10:51.173026 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b4acc3-8d21-4d60-9e32-f0cb1c2ed779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:10:51.199347 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f4b4acc3-8d21-4d60-9e32-f0cb1c2ed779 None None] MFA Rules not processed for user `08bc0ca115c8459792f49e96eb67f5d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:10:51.200540 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f4b4acc3-8d21-4d60-9e32-f0cb1c2ed779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:10:51.264836 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 760/1502] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:10:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:10:51.278622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3903ac81-10b2-4287-82ea-6ac7752ab79e None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:10:51.278955 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3903ac81-10b2-4287-82ea-6ac7752ab79e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:10:51.279242 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3903ac81-10b2-4287-82ea-6ac7752ab79e None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:10:51.280616 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 743/1503] 10.222.0.50 () {54 vars in 963 bytes} [Fri May 27 15:10:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:10:51.285547 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a0bd24-fd12-4022-93c9-03e344d95923 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:10:51.285838 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a0bd24-fd12-4022-93c9-03e344d95923 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:10:51.286120 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a0bd24-fd12-4022-93c9-03e344d95923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:10:51.305281 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-59a0bd24-fd12-4022-93c9-03e344d95923 None None] MFA Rules not processed for user `896af6ccf89847e5903f6a4fe3d9d810`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:10:51.306494 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-59a0bd24-fd12-4022-93c9-03e344d95923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:10:51.351398 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:10:51.359866 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 761/1504] 10.222.0.50 () {58 vars in 1060 bytes} [Fri May 27 15:10:51 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:10:51.367910 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3903ac81-10b2-4287-82ea-6ac7752ab79e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:10:51.378424 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3903ac81-10b2-4287-82ea-6ac7752ab79e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:10:51.382344 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b1cbc5-2e7e-4ccb-88e9-b6952690aa8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:10:51.382997 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1cbc5-2e7e-4ccb-88e9-b6952690aa8f service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:10:51.383279 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1cbc5-2e7e-4ccb-88e9-b6952690aa8f service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:10:51.383552 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1cbc5-2e7e-4ccb-88e9-b6952690aa8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:10:51.391685 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b1cbc5-2e7e-4ccb-88e9-b6952690aa8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:10:51.393116 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b1cbc5-2e7e-4ccb-88e9-b6952690aa8f service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:10:51.402396 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 744/1505] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:10:51 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:10:51.448683 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21d825-3b36-4ddb-b6d4-41b8f4ba45a8 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:10:51.448969 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21d825-3b36-4ddb-b6d4-41b8f4ba45a8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:10:51.449243 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21d825-3b36-4ddb-b6d4-41b8f4ba45a8 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:10:51.450577 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 762/1506] 10.222.0.50 () {52 vars in 829 bytes} [Fri May 27 15:10:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:10:51.454135 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b1cbc5-2e7e-4ccb-88e9-b6952690aa8f service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:10:51.515358 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b1cbc5-2e7e-4ccb-88e9-b6952690aa8f service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:10:51.520054 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707a9b30-cc2f-4a9e-9920-417d31fb9d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:10:51.520910 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a9b30-cc2f-4a9e-9920-417d31fb9d7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:10:51.521186 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a9b30-cc2f-4a9e-9920-417d31fb9d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:10:51.521456 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707a9b30-cc2f-4a9e-9920-417d31fb9d7f admin admin] PATH_INFO: `/v3/users/08bc0ca115c8459792f49e96eb67f5d5` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:10:51.522610 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707a9b30-cc2f-4a9e-9920-417d31fb9d7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=08bc0ca115c8459792f49e96eb67f5d5)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:10:51.523649 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: WARNING py.warnings [None req-707a9b30-cc2f-4a9e-9920-417d31fb9d7f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 15:10:51.523649 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: warnings.warn(msg) May 27 15:10:51.523649 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: May 27 15:10:51.524828 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707a9b30-cc2f-4a9e-9920-417d31fb9d7f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:10:51.590218 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-707a9b30-cc2f-4a9e-9920-417d31fb9d7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08bc0ca115c8459792f49e96eb67f5d5'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:10:51.600602 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-707a9b30-cc2f-4a9e-9920-417d31fb9d7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08bc0ca115c8459792f49e96eb67f5d5'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:10:51.612866 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-707a9b30-cc2f-4a9e-9920-417d31fb9d7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08bc0ca115c8459792f49e96eb67f5d5'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:10:51.614016 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-707a9b30-cc2f-4a9e-9920-417d31fb9d7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08bc0ca115c8459792f49e96eb67f5d5'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:10:51.614714 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-707a9b30-cc2f-4a9e-9920-417d31fb9d7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08bc0ca115c8459792f49e96eb67f5d5'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:10:51.631100 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:10:51.640957 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 745/1507] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:10:51 2022] DELETE /identity/v3/users/08bc0ca115c8459792f49e96eb67f5d5 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:10:51.645362 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a8784-4285-4c27-aa0f-117a7fd879b7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:10:51.645657 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a8784-4285-4c27-aa0f-117a7fd879b7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:10:51.645942 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a8784-4285-4c27-aa0f-117a7fd879b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:10:51.663609 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d45a8784-4285-4c27-aa0f-117a7fd879b7 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:10:51.664801 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d45a8784-4285-4c27-aa0f-117a7fd879b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:10:51.722696 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 763/1508] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:10:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:10:51.732864 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707a9b30-cc2f-4a9e-9920-417d31fb9d7f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:10:51.734298 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-707a9b30-cc2f-4a9e-9920-417d31fb9d7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:10:51.791012 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707a9b30-cc2f-4a9e-9920-417d31fb9d7f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:10:51.794676 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ea48cd-5fcd-4112-948b-e5d8d9bc9a5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:10:51.795356 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea48cd-5fcd-4112-948b-e5d8d9bc9a5c service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:10:51.795639 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea48cd-5fcd-4112-948b-e5d8d9bc9a5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:10:51.795919 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ea48cd-5fcd-4112-948b-e5d8d9bc9a5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:10:51.805893 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea48cd-5fcd-4112-948b-e5d8d9bc9a5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:10:51.807388 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ea48cd-5fcd-4112-948b-e5d8d9bc9a5c service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:10:51.816738 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 746/1509] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:10:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:10:52.168657 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a8784-4285-4c27-aa0f-117a7fd879b7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:10:52.170249 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d45a8784-4285-4c27-aa0f-117a7fd879b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:10:52.182820 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a8784-4285-4c27-aa0f-117a7fd879b7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:10:52.188824 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55e4b83-6576-40f2-b963-b133374e5f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:10:52.189921 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e4b83-6576-40f2-b963-b133374e5f8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:10:52.190278 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e4b83-6576-40f2-b963-b133374e5f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:10:52.190445 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55e4b83-6576-40f2-b963-b133374e5f8d admin admin] PATH_INFO: `/v3/projects/ca13243dae1546d49c6b2bce160850fb` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:10:52.191674 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55e4b83-6576-40f2-b963-b133374e5f8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca13243dae1546d49c6b2bce160850fb)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:10:52.193053 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: WARNING py.warnings [None req-a55e4b83-6576-40f2-b963-b133374e5f8d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 15:10:52.193053 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: warnings.warn(msg) May 27 15:10:52.193053 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: May 27 15:10:52.194519 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55e4b83-6576-40f2-b963-b133374e5f8d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:10:52.256106 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a55e4b83-6576-40f2-b963-b133374e5f8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca13243dae1546d49c6b2bce160850fb', 'request_id': 'req-a55e4b83-6576-40f2-b963-b133374e5f8d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:10:52.265011 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a55e4b83-6576-40f2-b963-b133374e5f8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca13243dae1546d49c6b2bce160850fb', 'request_id': 'req-a55e4b83-6576-40f2-b963-b133374e5f8d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:10:52.271885 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a55e4b83-6576-40f2-b963-b133374e5f8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca13243dae1546d49c6b2bce160850fb', 'request_id': 'req-a55e4b83-6576-40f2-b963-b133374e5f8d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:10:52.294704 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a55e4b83-6576-40f2-b963-b133374e5f8d admin admin] The token cache is being invalidate because project ca13243dae1546d49c6b2bce160850fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:10:52.295023 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a55e4b83-6576-40f2-b963-b133374e5f8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:10:52.295844 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a55e4b83-6576-40f2-b963-b133374e5f8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:10:52.297349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 764/1510] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:10:52 2022] DELETE /identity/v3/projects/ca13243dae1546d49c6b2bce160850fb => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:11:01.812501 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:11:02.287599 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:12:37.143603 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1dccad-9b88-4a41-bb99-b78370d1bd1f None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:37.144175 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1dccad-9b88-4a41-bb99-b78370d1bd1f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:37.144175 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1dccad-9b88-4a41-bb99-b78370d1bd1f None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:37.145501 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 747/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 15:12:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:12:37.151455 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50737f90-8395-4d11-ade5-82f95469ed65 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:37.151994 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50737f90-8395-4d11-ade5-82f95469ed65 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:37.151994 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50737f90-8395-4d11-ade5-82f95469ed65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:37.170721 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-50737f90-8395-4d11-ade5-82f95469ed65 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:37.171960 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-50737f90-8395-4d11-ade5-82f95469ed65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:37.232564 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 765/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 15:12:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:37.400240 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1dccad-9b88-4a41-bb99-b78370d1bd1f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:37.401946 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fc1dccad-9b88-4a41-bb99-b78370d1bd1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:37.474515 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1dccad-9b88-4a41-bb99-b78370d1bd1f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:37.480717 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ebacd-fc0d-4f00-af00-24e194e107a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '451a9232f95e494683d81b6078a318d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:37.481357 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ebacd-fc0d-4f00-af00-24e194e107a9 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:37.481636 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ebacd-fc0d-4f00-af00-24e194e107a9 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:37.481908 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ebacd-fc0d-4f00-af00-24e194e107a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:37.489826 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024ebacd-fc0d-4f00-af00-24e194e107a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:37.491496 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024ebacd-fc0d-4f00-af00-24e194e107a9 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:37.500476 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 748/1513] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:12:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:12:38.106388 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc471d4-e4dc-4f19-9009-73577fe99a1a None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:38.106686 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc471d4-e4dc-4f19-9009-73577fe99a1a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:38.106994 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc471d4-e4dc-4f19-9009-73577fe99a1a None None] PATH_INFO: `/v3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:38.108316 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 766/1514] 10.222.0.27 () {54 vars in 925 bytes} [Fri May 27 15:12:38 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:12:38.114748 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472cbd1-9fd5-4b7f-a73a-2b4610a61635 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:38.115126 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472cbd1-9fd5-4b7f-a73a-2b4610a61635 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:38.116004 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4472cbd1-9fd5-4b7f-a73a-2b4610a61635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:38.134663 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4472cbd1-9fd5-4b7f-a73a-2b4610a61635 None None] MFA Rules not processed for user `8293dab3ffd647f5927d323572fdbdce`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:38.135939 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4472cbd1-9fd5-4b7f-a73a-2b4610a61635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:38.202121 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 749/1515] 10.222.0.27 () {58 vars in 1015 bytes} [Fri May 27 15:12:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:38.388349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f916fc8-c7bb-4faf-9203-1afa9910d798 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:38.388884 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f916fc8-c7bb-4faf-9203-1afa9910d798 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:38.388884 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f916fc8-c7bb-4faf-9203-1afa9910d798 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:38.390235 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 767/1516] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 15:12:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:12:38.395061 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfcd728-d0b4-4384-bf49-8532838334fd None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:38.395354 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfcd728-d0b4-4384-bf49-8532838334fd None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:38.395632 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfcd728-d0b4-4384-bf49-8532838334fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:38.413817 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-bcfcd728-d0b4-4384-bf49-8532838334fd None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:38.415025 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bcfcd728-d0b4-4384-bf49-8532838334fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:38.422813 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 750/1517] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 15:12:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:38.574798 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f916fc8-c7bb-4faf-9203-1afa9910d798 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:38.589536 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f916fc8-c7bb-4faf-9203-1afa9910d798 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:38.595131 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0ee6e2-7cd4-423a-bd5e-06fbb9f9ad09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '451a9232f95e494683d81b6078a318d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:38.595760 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ee6e2-7cd4-423a-bd5e-06fbb9f9ad09 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:38.596048 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ee6e2-7cd4-423a-bd5e-06fbb9f9ad09 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:38.596327 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ee6e2-7cd4-423a-bd5e-06fbb9f9ad09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:38.604616 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0ee6e2-7cd4-423a-bd5e-06fbb9f9ad09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:38.606202 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0ee6e2-7cd4-423a-bd5e-06fbb9f9ad09 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:38.616067 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 768/1518] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:12:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:12:47.292078 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8228491e-cba1-4cd5-be29-46d41b9a1a33 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:47.292669 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8228491e-cba1-4cd5-be29-46d41b9a1a33 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:47.292669 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8228491e-cba1-4cd5-be29-46d41b9a1a33 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:47.294024 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 751/1519] 10.222.0.50 () {54 vars in 960 bytes} [Fri May 27 15:12:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:12:47.299073 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ceedf2-5b68-44fa-b5bc-33ca0a24516a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:47.299498 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ceedf2-5b68-44fa-b5bc-33ca0a24516a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:47.299666 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ceedf2-5b68-44fa-b5bc-33ca0a24516a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:47.318329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-38ceedf2-5b68-44fa-b5bc-33ca0a24516a None None] MFA Rules not processed for user `8293dab3ffd647f5927d323572fdbdce`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:47.319600 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-38ceedf2-5b68-44fa-b5bc-33ca0a24516a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:47.328047 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 769/1520] 10.222.0.50 () {58 vars in 1057 bytes} [Fri May 27 15:12:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:47.335708 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8228491e-cba1-4cd5-be29-46d41b9a1a33 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:47.349154 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8228491e-cba1-4cd5-be29-46d41b9a1a33 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:47.354643 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30de7e79-151a-4b46-b271-6e3d8442b4ac service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8293dab3ffd647f5927d323572fdbdce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:47.355298 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de7e79-151a-4b46-b271-6e3d8442b4ac service placement] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:47.355586 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de7e79-151a-4b46-b271-6e3d8442b4ac service placement] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:47.355868 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de7e79-151a-4b46-b271-6e3d8442b4ac service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:47.363987 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30de7e79-151a-4b46-b271-6e3d8442b4ac service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:47.365588 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30de7e79-151a-4b46-b271-6e3d8442b4ac service placement] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:47.375692 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 752/1521] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:12:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:12:49.915952 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f19b3-f824-4fa1-9a88-eb9629aaa1cb None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:49.916610 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f19b3-f824-4fa1-9a88-eb9629aaa1cb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:49.916610 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5f19b3-f824-4fa1-9a88-eb9629aaa1cb None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:49.917902 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 770/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 15:12:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 27 15:12:49.922935 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea1328-a677-4804-80b1-80ad25350d07 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:49.923363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea1328-a677-4804-80b1-80ad25350d07 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:49.923493 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea1328-a677-4804-80b1-80ad25350d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:49.941839 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-60ea1328-a677-4804-80b1-80ad25350d07 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:49.943098 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-60ea1328-a677-4804-80b1-80ad25350d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:49.951436 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 753/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 15:12:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:50.105139 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5f19b3-f824-4fa1-9a88-eb9629aaa1cb None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:50.119886 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5f19b3-f824-4fa1-9a88-eb9629aaa1cb None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:50.125182 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e639f07-af44-44da-a4a6-8936b1a739f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '451a9232f95e494683d81b6078a318d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:50.125816 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e639f07-af44-44da-a4a6-8936b1a739f1 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:50.126099 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e639f07-af44-44da-a4a6-8936b1a739f1 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:50.126374 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e639f07-af44-44da-a4a6-8936b1a739f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:50.135024 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e639f07-af44-44da-a4a6-8936b1a739f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:50.136603 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e639f07-af44-44da-a4a6-8936b1a739f1 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:50.145772 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 771/1524] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:12:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:12:51.027242 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289168d3-cf2b-4fa2-9ff1-fff4655c5ca2 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:51.027812 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289168d3-cf2b-4fa2-9ff1-fff4655c5ca2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:51.027812 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289168d3-cf2b-4fa2-9ff1-fff4655c5ca2 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:51.029190 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 754/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 27 15:12:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:12:51.033907 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a67fa-a3e3-4d86-ac63-ba40f83a5252 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:51.034309 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a67fa-a3e3-4d86-ac63-ba40f83a5252 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:51.034468 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a67fa-a3e3-4d86-ac63-ba40f83a5252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:51.052332 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-774a67fa-a3e3-4d86-ac63-ba40f83a5252 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:51.053514 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-774a67fa-a3e3-4d86-ac63-ba40f83a5252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:51.061443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 772/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 27 15:12:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:51.210685 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289168d3-cf2b-4fa2-9ff1-fff4655c5ca2 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:51.225429 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289168d3-cf2b-4fa2-9ff1-fff4655c5ca2 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:51.230929 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f77304-4fc6-48e9-af1c-e9657a7acf7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '451a9232f95e494683d81b6078a318d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:51.231567 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f77304-4fc6-48e9-af1c-e9657a7acf7b service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:51.231851 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f77304-4fc6-48e9-af1c-e9657a7acf7b service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:51.232131 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f77304-4fc6-48e9-af1c-e9657a7acf7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:51.240451 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f77304-4fc6-48e9-af1c-e9657a7acf7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:51.242022 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f77304-4fc6-48e9-af1c-e9657a7acf7b service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:51.251390 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 755/1527] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:12:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:12:55.681572 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5113cd4-164c-4bff-8f4b-4441646bd296 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:55.682882 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5113cd4-164c-4bff-8f4b-4441646bd296 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:55.682882 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5113cd4-164c-4bff-8f4b-4441646bd296 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:55.684095 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 773/1528] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 15:12:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:12:55.689431 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75206f69-2f8a-46f1-bef7-5c47f8699cab None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:55.689862 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75206f69-2f8a-46f1-bef7-5c47f8699cab None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:55.689994 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75206f69-2f8a-46f1-bef7-5c47f8699cab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:55.707916 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-75206f69-2f8a-46f1-bef7-5c47f8699cab None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:55.709092 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-75206f69-2f8a-46f1-bef7-5c47f8699cab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:55.716905 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 756/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 15:12:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:55.753776 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5113cd4-164c-4bff-8f4b-4441646bd296 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:55.755501 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c5113cd4-164c-4bff-8f4b-4441646bd296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:55.813206 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5113cd4-164c-4bff-8f4b-4441646bd296 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:55.816741 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb236330-2be6-4228-bc14-6cfaf8df7e15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:55.817385 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb236330-2be6-4228-bc14-6cfaf8df7e15 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:55.817669 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb236330-2be6-4228-bc14-6cfaf8df7e15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:55.817949 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb236330-2be6-4228-bc14-6cfaf8df7e15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:55.827867 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb236330-2be6-4228-bc14-6cfaf8df7e15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:55.829290 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb236330-2be6-4228-bc14-6cfaf8df7e15 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:55.839224 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 774/1530] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:12:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:12:58.051568 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57576faa-a6ac-4291-8aa6-0c5ac483c156 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:58.052145 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57576faa-a6ac-4291-8aa6-0c5ac483c156 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:58.052145 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57576faa-a6ac-4291-8aa6-0c5ac483c156 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:58.053491 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 757/1531] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 15:12:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:12:58.059327 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a35032-cbac-4113-aef5-5e4150baac16 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:58.059789 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a35032-cbac-4113-aef5-5e4150baac16 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:58.059897 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a35032-cbac-4113-aef5-5e4150baac16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:58.080782 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d8a35032-cbac-4113-aef5-5e4150baac16 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:12:58.081979 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d8a35032-cbac-4113-aef5-5e4150baac16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:58.089977 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 775/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 15:12:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:12:58.126370 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57576faa-a6ac-4291-8aa6-0c5ac483c156 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:12:58.128092 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-57576faa-a6ac-4291-8aa6-0c5ac483c156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:12:58.141629 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57576faa-a6ac-4291-8aa6-0c5ac483c156 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:12:58.145085 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edeb5a18-9d37-49ce-a72b-329391e0ffdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:12:58.145719 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb5a18-9d37-49ce-a72b-329391e0ffdb service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:12:58.146001 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb5a18-9d37-49ce-a72b-329391e0ffdb service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:12:58.146277 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edeb5a18-9d37-49ce-a72b-329391e0ffdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:12:58.155340 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeb5a18-9d37-49ce-a72b-329391e0ffdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:12:58.156779 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edeb5a18-9d37-49ce-a72b-329391e0ffdb service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:12:58.166201 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 758/1533] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:12:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:13:01.993340 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240e69f-39d4-488a-b88a-f62ad90575b3 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:01.994002 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240e69f-39d4-488a-b88a-f62ad90575b3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:01.994002 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240e69f-39d4-488a-b88a-f62ad90575b3 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:01.996046 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 776/1534] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 15:13:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:13:02.001796 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de535ecb-fd10-42fa-8d26-150dbede725f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:02.002272 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de535ecb-fd10-42fa-8d26-150dbede725f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:02.002373 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de535ecb-fd10-42fa-8d26-150dbede725f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:02.008626 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=91 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:02.021943 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-de535ecb-fd10-42fa-8d26-150dbede725f None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:02.023169 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-de535ecb-fd10-42fa-8d26-150dbede725f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:02.031036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 759/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 15:13:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:02.309076 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2240e69f-39d4-488a-b88a-f62ad90575b3 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:02.310792 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2240e69f-39d4-488a-b88a-f62ad90575b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:02.316733 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=75 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:02.370005 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2240e69f-39d4-488a-b88a-f62ad90575b3 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:02.373552 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e21943-bf9f-4aef-8f30-30efc1d55b1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:02.374191 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e21943-bf9f-4aef-8f30-30efc1d55b1a service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:02.374479 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e21943-bf9f-4aef-8f30-30efc1d55b1a service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:02.374757 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e21943-bf9f-4aef-8f30-30efc1d55b1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:02.383561 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e21943-bf9f-4aef-8f30-30efc1d55b1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:02.385094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e21943-bf9f-4aef-8f30-30efc1d55b1a service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:02.394203 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 777/1536] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:13:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:13:04.659730 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f74486-3d54-40fc-9626-8b50268f377c None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:04.660342 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f74486-3d54-40fc-9626-8b50268f377c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:04.660342 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f74486-3d54-40fc-9626-8b50268f377c None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:04.661668 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 760/1537] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 27 15:13:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:13:04.668258 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d6aa3-4d85-4696-a20c-b5618f7ab1cd None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:04.668843 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d6aa3-4d85-4696-a20c-b5618f7ab1cd None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:04.668843 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d6aa3-4d85-4696-a20c-b5618f7ab1cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:04.690142 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2a5d6aa3-4d85-4696-a20c-b5618f7ab1cd None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:13:04.691424 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2a5d6aa3-4d85-4696-a20c-b5618f7ab1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:04.704283 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 778/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 27 15:13:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:13:04.979828 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f74486-3d54-40fc-9626-8b50268f377c None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:13:04.981932 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-20f74486-3d54-40fc-9626-8b50268f377c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:13:05.000673 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f74486-3d54-40fc-9626-8b50268f377c None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:13:05.004337 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545ff596-cdf7-4ab6-a9af-1e750eada87c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:13:05.005076 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545ff596-cdf7-4ab6-a9af-1e750eada87c service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:13:05.005572 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545ff596-cdf7-4ab6-a9af-1e750eada87c service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:13:05.006022 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545ff596-cdf7-4ab6-a9af-1e750eada87c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:13:05.016341 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545ff596-cdf7-4ab6-a9af-1e750eada87c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:13:05.017966 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545ff596-cdf7-4ab6-a9af-1e750eada87c service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:13:05.028956 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 761/1539] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:13:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:13:14.683721 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=28 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:13:15.023485 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=8 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:37.560939 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff6cc3-b162-4804-b55e-0eaaa73163c3 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:37.561934 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff6cc3-b162-4804-b55e-0eaaa73163c3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:37.562385 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff6cc3-b162-4804-b55e-0eaaa73163c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:37.603233 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-51ff6cc3-b162-4804-b55e-0eaaa73163c3 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:37.604803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-51ff6cc3-b162-4804-b55e-0eaaa73163c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:37.618936 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 779/1540] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:14:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:14:37.630349 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4baca-5833-43b1-833c-548fcb56663e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:37.631317 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4baca-5833-43b1-833c-548fcb56663e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:37.631785 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b4baca-5833-43b1-833c-548fcb56663e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:37.641682 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d398457-7978-4284-905e-db6d4e314e20 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:37.642478 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d398457-7978-4284-905e-db6d4e314e20 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:37.643897 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d398457-7978-4284-905e-db6d4e314e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:37.654394 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-85b4baca-5833-43b1-833c-548fcb56663e None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:37.662877 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-85b4baca-5833-43b1-833c-548fcb56663e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:37.667742 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9d398457-7978-4284-905e-db6d4e314e20 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:37.669024 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9d398457-7978-4284-905e-db6d4e314e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:37.672653 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 762/1541] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:14:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:14:37.674330 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4290dc-6375-49aa-ad03-b4fe8d78c633 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:37.674842 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4290dc-6375-49aa-ad03-b4fe8d78c633 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:37.675351 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4290dc-6375-49aa-ad03-b4fe8d78c633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:37.696383 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 780/1542] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:14:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:37.697593 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d398457-7978-4284-905e-db6d4e314e20 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:37.708229 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4a4290dc-6375-49aa-ad03-b4fe8d78c633 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:37.709783 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4a4290dc-6375-49aa-ad03-b4fe8d78c633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:37.715554 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d398457-7978-4284-905e-db6d4e314e20 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:37.719675 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 763/1543] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:14:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:37.721203 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136cfa09-e989-4de6-82a7-272f0b44a93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:37.721694 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b8102c-8ae9-4099-88ec-56e863f57614 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:37.721884 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136cfa09-e989-4de6-82a7-272f0b44a93c admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:37.722269 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b8102c-8ae9-4099-88ec-56e863f57614 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:37.722542 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136cfa09-e989-4de6-82a7-272f0b44a93c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:37.722676 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b8102c-8ae9-4099-88ec-56e863f57614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:37.723129 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136cfa09-e989-4de6-82a7-272f0b44a93c admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:37.724113 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136cfa09-e989-4de6-82a7-272f0b44a93c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:14:37.724578 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136cfa09-e989-4de6-82a7-272f0b44a93c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:37.726358 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136cfa09-e989-4de6-82a7-272f0b44a93c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:37.742112 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 781/1544] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:14:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:14:37.742362 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d9b8102c-8ae9-4099-88ec-56e863f57614 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:37.743826 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d9b8102c-8ae9-4099-88ec-56e863f57614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:37.743975 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510f6e5-b3d2-4abb-a494-8e076377859f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:37.744302 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510f6e5-b3d2-4abb-a494-8e076377859f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:37.750893 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5510f6e5-b3d2-4abb-a494-8e076377859f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:37.760492 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 764/1545] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:14:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:37.762195 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef5a5b-b22a-450e-a6af-fcb33d92c5b5 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:37.763243 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef5a5b-b22a-450e-a6af-fcb33d92c5b5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:37.763694 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef5a5b-b22a-450e-a6af-fcb33d92c5b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:37.765771 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5510f6e5-b3d2-4abb-a494-8e076377859f None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:37.767291 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5510f6e5-b3d2-4abb-a494-8e076377859f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:37.775862 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 782/1546] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:14:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:37.782873 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc0006-afe5-4855-bebd-11600b6ecba7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:37.782873 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc0006-afe5-4855-bebd-11600b6ecba7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:37.782873 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cc0006-afe5-4855-bebd-11600b6ecba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:37.791641 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-72ef5a5b-b22a-450e-a6af-fcb33d92c5b5 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:37.793134 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-72ef5a5b-b22a-450e-a6af-fcb33d92c5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:37.797188 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-00cc0006-afe5-4855-bebd-11600b6ecba7 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:37.798468 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-00cc0006-afe5-4855-bebd-11600b6ecba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:37.802262 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 765/1547] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:14:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:37.803827 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ecd00-eba0-4001-a480-fc275ac41a51 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:37.804331 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ecd00-eba0-4001-a480-fc275ac41a51 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:37.804803 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ecd00-eba0-4001-a480-fc275ac41a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:37.806895 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 783/1548] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:14:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:37.807842 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cc0006-afe5-4855-bebd-11600b6ecba7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:37.818573 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cc0006-afe5-4855-bebd-11600b6ecba7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:37.823224 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3a9039-1fc2-4bfd-8c02-3c5b604ac2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:37.823897 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a9039-1fc2-4bfd-8c02-3c5b604ac2e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:37.824186 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a9039-1fc2-4bfd-8c02-3c5b604ac2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:37.824469 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3a9039-1fc2-4bfd-8c02-3c5b604ac2e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:37.825754 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3a9039-1fc2-4bfd-8c02-3c5b604ac2e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:37.826923 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: WARNING py.warnings [None req-4b3a9039-1fc2-4bfd-8c02-3c5b604ac2e1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 15:14:37.826923 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: warnings.warn(msg) May 27 15:14:37.826923 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: May 27 15:14:37.827878 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d39ecd00-eba0-4001-a480-fc275ac41a51 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:37.828126 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3a9039-1fc2-4bfd-8c02-3c5b604ac2e1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:37.829515 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d39ecd00-eba0-4001-a480-fc275ac41a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:37.840606 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 766/1549] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:14:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:37.842230 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1358097-4de9-4a95-a50a-5bef763d77a2 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:37.843031 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1358097-4de9-4a95-a50a-5bef763d77a2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:37.843542 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1358097-4de9-4a95-a50a-5bef763d77a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:37.860850 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 784/1550] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:14:37 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:14:37.862732 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade8fce-a31a-4c3a-b482-575dfa4eaba7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:37.863247 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade8fce-a31a-4c3a-b482-575dfa4eaba7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:37.863738 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade8fce-a31a-4c3a-b482-575dfa4eaba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:37.864383 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c1358097-4de9-4a95-a50a-5bef763d77a2 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:37.865565 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c1358097-4de9-4a95-a50a-5bef763d77a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:37.883034 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3ade8fce-a31a-4c3a-b482-575dfa4eaba7 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:37.884231 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3ade8fce-a31a-4c3a-b482-575dfa4eaba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:37.943326 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 767/1551] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:14:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:37.944670 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1358097-4de9-4a95-a50a-5bef763d77a2 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:37.959276 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1358097-4de9-4a95-a50a-5bef763d77a2 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:37.960283 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 785/1552] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:14:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:37.961397 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64246fee-49ad-4a87-bad2-7afb3f1a597a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:37.961686 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64246fee-49ad-4a87-bad2-7afb3f1a597a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:37.961961 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64246fee-49ad-4a87-bad2-7afb3f1a597a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:37.968963 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e1a745-2878-4e35-8d06-aa0c4bd79ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:37.969710 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1a745-2878-4e35-8d06-aa0c4bd79ad6 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:37.970142 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1a745-2878-4e35-8d06-aa0c4bd79ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:37.970556 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1a745-2878-4e35-8d06-aa0c4bd79ad6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:37.971725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e1a745-2878-4e35-8d06-aa0c4bd79ad6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:14:37.972187 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e1a745-2878-4e35-8d06-aa0c4bd79ad6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:37.973862 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: WARNING py.warnings [None req-69e1a745-2878-4e35-8d06-aa0c4bd79ad6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 27 15:14:37.973862 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: warnings.warn(msg) May 27 15:14:37.973862 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: May 27 15:14:37.975061 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e1a745-2878-4e35-8d06-aa0c4bd79ad6 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:37.982189 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-64246fee-49ad-4a87-bad2-7afb3f1a597a None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:37.983431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-64246fee-49ad-4a87-bad2-7afb3f1a597a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:37.987700 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 768/1553] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:14:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:14:37.989590 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e199bd6-209c-4c32-9464-98e092fe4316 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:37.989884 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e199bd6-209c-4c32-9464-98e092fe4316 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:37.990163 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e199bd6-209c-4c32-9464-98e092fe4316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:37.996223 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 786/1554] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:14:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:37.997641 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64246fee-49ad-4a87-bad2-7afb3f1a597a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:38.007075 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64246fee-49ad-4a87-bad2-7afb3f1a597a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:38.011920 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c965ac8-bfb2-489b-b330-1ca43090f879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:38.013053 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c965ac8-bfb2-489b-b330-1ca43090f879 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:38.013342 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c965ac8-bfb2-489b-b330-1ca43090f879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:38.013624 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c965ac8-bfb2-489b-b330-1ca43090f879 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:38.013739 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5e199bd6-209c-4c32-9464-98e092fe4316 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:38.014351 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c965ac8-bfb2-489b-b330-1ca43090f879 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:14:38.014659 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c965ac8-bfb2-489b-b330-1ca43090f879 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:38.014927 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5e199bd6-209c-4c32-9464-98e092fe4316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:38.016325 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c965ac8-bfb2-489b-b330-1ca43090f879 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:38.024103 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 769/1555] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:14:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:38.025807 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6f844-306e-49a5-9045-a7344e0fce88 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:38.026269 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6f844-306e-49a5-9045-a7344e0fce88 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:38.026555 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc6f844-306e-49a5-9045-a7344e0fce88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:38.028087 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 787/1556] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:14:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:14:38.029178 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c965ac8-bfb2-489b-b330-1ca43090f879 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:38.039060 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c965ac8-bfb2-489b-b330-1ca43090f879 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:38.044065 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb95bcda-c397-4a63-a318-fbd4676140c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:38.044886 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95bcda-c397-4a63-a318-fbd4676140c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:38.045173 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95bcda-c397-4a63-a318-fbd4676140c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:38.045452 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb95bcda-c397-4a63-a318-fbd4676140c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:38.046132 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb95bcda-c397-4a63-a318-fbd4676140c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:38.048134 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb95bcda-c397-4a63-a318-fbd4676140c2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:38.049548 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3fc6f844-306e-49a5-9045-a7344e0fce88 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:38.050739 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3fc6f844-306e-49a5-9045-a7344e0fce88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:38.060803 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 770/1557] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:14:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:38.067191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc6f844-306e-49a5-9045-a7344e0fce88 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:38.070456 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 788/1558] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:14:38 2022] POST /identity/v3/users => generated 289 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:14:38.071590 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb95bcda-c397-4a63-a318-fbd4676140c2 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:38.073303 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc6f844-306e-49a5-9045-a7344e0fce88 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:38.078055 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2857635b-88db-4faf-bfdc-42ab8194663e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:38.078712 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2857635b-88db-4faf-bfdc-42ab8194663e admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:38.079028 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2857635b-88db-4faf-bfdc-42ab8194663e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:38.079314 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2857635b-88db-4faf-bfdc-42ab8194663e admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:38.080041 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2857635b-88db-4faf-bfdc-42ab8194663e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:14:38.080355 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2857635b-88db-4faf-bfdc-42ab8194663e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:38.081505 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb95bcda-c397-4a63-a318-fbd4676140c2 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:38.082053 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2857635b-88db-4faf-bfdc-42ab8194663e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:38.086044 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b896ba0-5b36-43c5-a077-15174c66226e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:38.086710 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b896ba0-5b36-43c5-a077-15174c66226e admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:38.087019 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b896ba0-5b36-43c5-a077-15174c66226e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:38.087295 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b896ba0-5b36-43c5-a077-15174c66226e admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:38.087999 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b896ba0-5b36-43c5-a077-15174c66226e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:14:38.088300 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b896ba0-5b36-43c5-a077-15174c66226e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:38.089870 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b896ba0-5b36-43c5-a077-15174c66226e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:38.094059 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 771/1559] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:14:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:14:38.094880 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2857635b-88db-4faf-bfdc-42ab8194663e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:38.112214 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 789/1560] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:14:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:14:38.112400 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2857635b-88db-4faf-bfdc-42ab8194663e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:38.112531 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b896ba0-5b36-43c5-a077-15174c66226e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:38.112642 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c098f0-ac86-4a71-a8bb-05961c357df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:38.112642 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c098f0-ac86-4a71-a8bb-05961c357df1 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:38.112642 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c098f0-ac86-4a71-a8bb-05961c357df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:38.112642 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c098f0-ac86-4a71-a8bb-05961c357df1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:38.112996 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c098f0-ac86-4a71-a8bb-05961c357df1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:38.113935 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c098f0-ac86-4a71-a8bb-05961c357df1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:38.118074 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b896ba0-5b36-43c5-a077-15174c66226e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:38.123476 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb039984-dc14-4fce-895e-46f4332f8315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:38.124127 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb039984-dc14-4fce-895e-46f4332f8315 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:38.124405 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb039984-dc14-4fce-895e-46f4332f8315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:38.124685 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb039984-dc14-4fce-895e-46f4332f8315 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:38.125399 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb039984-dc14-4fce-895e-46f4332f8315 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:14:38.125703 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb039984-dc14-4fce-895e-46f4332f8315 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:38.127318 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb039984-dc14-4fce-895e-46f4332f8315 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:38.140316 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 790/1561] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:14:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:14:38.141577 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8922bd9-0ee2-4dde-9db6-8720cff26b4d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:38.141715 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 772/1562] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:14:38 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:14:38.142109 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8922bd9-0ee2-4dde-9db6-8720cff26b4d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:38.142432 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8922bd9-0ee2-4dde-9db6-8720cff26b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:38.145568 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c098f0-ac86-4a71-a8bb-05961c357df1 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:38.169646 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d8922bd9-0ee2-4dde-9db6-8720cff26b4d None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:38.170868 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d8922bd9-0ee2-4dde-9db6-8720cff26b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:38.206743 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:38.211563 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c098f0-ac86-4a71-a8bb-05961c357df1 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:38.216813 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4144082-0e7e-4fcd-bf70-3c1489e070a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:38.217478 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4144082-0e7e-4fcd-bf70-3c1489e070a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:38.218019 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4144082-0e7e-4fcd-bf70-3c1489e070a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:38.218296 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4144082-0e7e-4fcd-bf70-3c1489e070a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:38.219049 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4144082-0e7e-4fcd-bf70-3c1489e070a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:38.220960 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4144082-0e7e-4fcd-bf70-3c1489e070a4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:38.230610 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:38.240401 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 791/1563] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:14:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:38.242074 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2878c51-ec34-44c4-8532-529980af4413 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:38.242403 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2878c51-ec34-44c4-8532-529980af4413 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:38.242792 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2878c51-ec34-44c4-8532-529980af4413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:38.258379 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 773/1564] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:14:38 2022] POST /identity/v3/projects => generated 373 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:14:38.258379 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4144082-0e7e-4fcd-bf70-3c1489e070a4 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:38.264147 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b2878c51-ec34-44c4-8532-529980af4413 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:38.265642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b2878c51-ec34-44c4-8532-529980af4413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:38.325413 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 792/1565] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:14:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:38.325919 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4144082-0e7e-4fcd-bf70-3c1489e070a4 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:38.326222 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2878c51-ec34-44c4-8532-529980af4413 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:38.332972 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a19e4f7-8776-475c-9d84-9f24d7e0281e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:38.333641 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a19e4f7-8776-475c-9d84-9f24d7e0281e admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:38.333927 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a19e4f7-8776-475c-9d84-9f24d7e0281e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:38.334208 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a19e4f7-8776-475c-9d84-9f24d7e0281e admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:38.335089 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a19e4f7-8776-475c-9d84-9f24d7e0281e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:38.335434 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2878c51-ec34-44c4-8532-529980af4413 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:38.336726 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a19e4f7-8776-475c-9d84-9f24d7e0281e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:38.340235 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab989d82-0e8d-4acc-99fd-ad895e1c7658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:38.341405 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab989d82-0e8d-4acc-99fd-ad895e1c7658 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:38.341680 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab989d82-0e8d-4acc-99fd-ad895e1c7658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:38.341948 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab989d82-0e8d-4acc-99fd-ad895e1c7658 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:38.342894 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab989d82-0e8d-4acc-99fd-ad895e1c7658 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:38.344794 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab989d82-0e8d-4acc-99fd-ad895e1c7658 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:38.347645 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 774/1566] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:14:38.348435 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a19e4f7-8776-475c-9d84-9f24d7e0281e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:38.358149 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a19e4f7-8776-475c-9d84-9f24d7e0281e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:38.362877 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be8604f-da2f-4561-8108-4224fbf7a362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:38.363940 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8604f-da2f-4561-8108-4224fbf7a362 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:38.364270 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8604f-da2f-4561-8108-4224fbf7a362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:38.364647 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8604f-da2f-4561-8108-4224fbf7a362 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:38.365561 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be8604f-da2f-4561-8108-4224fbf7a362 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:38.367664 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be8604f-da2f-4561-8108-4224fbf7a362 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:38.375506 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 793/1567] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:14:38 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:14:38.376322 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab989d82-0e8d-4acc-99fd-ad895e1c7658 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:38.402202 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 775/1568] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:14:38 2022] POST /identity/v3/projects => generated 403 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:14:38.402202 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be8604f-da2f-4561-8108-4224fbf7a362 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:38.439176 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab989d82-0e8d-4acc-99fd-ad895e1c7658 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:38.466354 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be8604f-da2f-4561-8108-4224fbf7a362 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:38.472074 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83ee36-8de7-49cb-9a94-6a828cbc713a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:38.472739 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83ee36-8de7-49cb-9a94-6a828cbc713a admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:38.473027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83ee36-8de7-49cb-9a94-6a828cbc713a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:38.473309 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b83ee36-8de7-49cb-9a94-6a828cbc713a admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:38.474055 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b83ee36-8de7-49cb-9a94-6a828cbc713a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:38.476021 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b83ee36-8de7-49cb-9a94-6a828cbc713a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:38.499238 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a66279-a8df-4662-89c5-6209bcb1cde7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:38.499910 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a66279-a8df-4662-89c5-6209bcb1cde7 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:38.500189 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a66279-a8df-4662-89c5-6209bcb1cde7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:38.500460 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a66279-a8df-4662-89c5-6209bcb1cde7 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:38.501145 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a66279-a8df-4662-89c5-6209bcb1cde7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:38.502651 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 776/1569] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:14:38 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:14:38.503097 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a66279-a8df-4662-89c5-6209bcb1cde7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:38.503513 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83ee36-8de7-49cb-9a94-6a828cbc713a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:38.510839 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:38.524494 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 794/1570] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:14:38 2022] POST /identity/v3/users => generated 283 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:14:38.525293 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a66279-a8df-4662-89c5-6209bcb1cde7 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:38.534599 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a66279-a8df-4662-89c5-6209bcb1cde7 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:38.563869 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:38.579118 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b83ee36-8de7-49cb-9a94-6a828cbc713a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:38.587219 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a57730-e273-4650-a598-59531a68067e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:38.587894 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a57730-e273-4650-a598-59531a68067e admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:38.588210 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a57730-e273-4650-a598-59531a68067e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:38.588491 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a57730-e273-4650-a598-59531a68067e admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:38.589340 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a57730-e273-4650-a598-59531a68067e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:14:38.589656 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a57730-e273-4650-a598-59531a68067e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:38.591469 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a57730-e273-4650-a598-59531a68067e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:38.603929 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 777/1571] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:14:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:14:38.605092 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a57730-e273-4650-a598-59531a68067e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:38.610176 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96429ff3-62c4-4d94-9e4b-7d6955eddc44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:38.610996 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96429ff3-62c4-4d94-9e4b-7d6955eddc44 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:38.611267 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96429ff3-62c4-4d94-9e4b-7d6955eddc44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:38.611550 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96429ff3-62c4-4d94-9e4b-7d6955eddc44 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:38.612248 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96429ff3-62c4-4d94-9e4b-7d6955eddc44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:38.614161 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96429ff3-62c4-4d94-9e4b-7d6955eddc44 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:38.617186 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a57730-e273-4650-a598-59531a68067e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:38.622453 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9077a5b7-909f-4e95-8e39-355d42e7b866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:38.623443 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9077a5b7-909f-4e95-8e39-355d42e7b866 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:38.623740 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9077a5b7-909f-4e95-8e39-355d42e7b866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:38.624027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9077a5b7-909f-4e95-8e39-355d42e7b866 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:38.624740 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9077a5b7-909f-4e95-8e39-355d42e7b866 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:14:38.625056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9077a5b7-909f-4e95-8e39-355d42e7b866 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:38.626672 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9077a5b7-909f-4e95-8e39-355d42e7b866 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:38.636230 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 795/1572] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:14:38 2022] POST /identity/v3/users => generated 287 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:14:38.640874 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96429ff3-62c4-4d94-9e4b-7d6955eddc44 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:38.642481 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 778/1573] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:14:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:14:38.643785 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9077a5b7-909f-4e95-8e39-355d42e7b866 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:38.653784 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9077a5b7-909f-4e95-8e39-355d42e7b866 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:38.654404 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96429ff3-62c4-4d94-9e4b-7d6955eddc44 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:38.658321 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e129249c-3b15-4c1c-b7d7-9ba1e64af707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:38.659139 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129249c-3b15-4c1c-b7d7-9ba1e64af707 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:38.659258 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfb3aea-3ba0-4cc8-8fd6-103fb6cc3a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:38.659429 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129249c-3b15-4c1c-b7d7-9ba1e64af707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:38.659709 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129249c-3b15-4c1c-b7d7-9ba1e64af707 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:38.659831 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb3aea-3ba0-4cc8-8fd6-103fb6cc3a64 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:38.660059 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb3aea-3ba0-4cc8-8fd6-103fb6cc3a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:38.660333 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb3aea-3ba0-4cc8-8fd6-103fb6cc3a64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:38.660439 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e129249c-3b15-4c1c-b7d7-9ba1e64af707 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:38.661050 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfb3aea-3ba0-4cc8-8fd6-103fb6cc3a64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:38.661429 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: WARNING py.warnings [None req-e129249c-3b15-4c1c-b7d7-9ba1e64af707 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 15:14:38.661429 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: warnings.warn(msg) May 27 15:14:38.661429 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: May 27 15:14:38.662642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfb3aea-3ba0-4cc8-8fd6-103fb6cc3a64 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:38.662808 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e129249c-3b15-4c1c-b7d7-9ba1e64af707 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:38.674840 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 796/1574] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:14:38.676008 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfb3aea-3ba0-4cc8-8fd6-103fb6cc3a64 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:38.685912 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 779/1575] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:14:38 2022] POST /identity/v3/users => generated 292 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:14:38.686931 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e129249c-3b15-4c1c-b7d7-9ba1e64af707 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:38.687061 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfb3aea-3ba0-4cc8-8fd6-103fb6cc3a64 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:38.691899 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a644d9e3-10b7-42e9-a6ac-7a57c70a7f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:38.692554 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644d9e3-10b7-42e9-a6ac-7a57c70a7f90 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:38.692839 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644d9e3-10b7-42e9-a6ac-7a57c70a7f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:38.693120 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644d9e3-10b7-42e9-a6ac-7a57c70a7f90 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:38.693800 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a644d9e3-10b7-42e9-a6ac-7a57c70a7f90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:38.695740 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a644d9e3-10b7-42e9-a6ac-7a57c70a7f90 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:38.696514 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e129249c-3b15-4c1c-b7d7-9ba1e64af707 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:38.701629 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2961e4f-649e-4dea-a9cf-aed68caf9dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:38.702268 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2961e4f-649e-4dea-a9cf-aed68caf9dd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:38.702568 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2961e4f-649e-4dea-a9cf-aed68caf9dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:38.702834 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2961e4f-649e-4dea-a9cf-aed68caf9dd1 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:38.703566 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2961e4f-649e-4dea-a9cf-aed68caf9dd1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:38.705508 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2961e4f-649e-4dea-a9cf-aed68caf9dd1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:38.718596 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 797/1576] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:14:38 2022] POST /identity/v3/users => generated 302 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:14:38.719588 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a644d9e3-10b7-42e9-a6ac-7a57c70a7f90 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:38.730153 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 780/1577] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:14:38 2022] POST /identity/v3/users => generated 294 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:14:38.730153 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2961e4f-649e-4dea-a9cf-aed68caf9dd1 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:38.733152 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a644d9e3-10b7-42e9-a6ac-7a57c70a7f90 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:38.737739 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3e63a5-86d1-48c3-9cbb-567ca10b0295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:38.738319 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2961e4f-649e-4dea-a9cf-aed68caf9dd1 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:38.738424 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e63a5-86d1-48c3-9cbb-567ca10b0295 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:38.738769 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e63a5-86d1-48c3-9cbb-567ca10b0295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:38.739069 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3e63a5-86d1-48c3-9cbb-567ca10b0295 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:38.739847 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3e63a5-86d1-48c3-9cbb-567ca10b0295 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:38.741430 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3e63a5-86d1-48c3-9cbb-567ca10b0295 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:38.742885 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e0e32-4b05-4bfa-bfd2-e2efaec0c205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:38.743526 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e0e32-4b05-4bfa-bfd2-e2efaec0c205 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:38.743812 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e0e32-4b05-4bfa-bfd2-e2efaec0c205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:38.744091 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e0e32-4b05-4bfa-bfd2-e2efaec0c205 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:38.744764 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45e0e32-4b05-4bfa-bfd2-e2efaec0c205 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:38.747233 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45e0e32-4b05-4bfa-bfd2-e2efaec0c205 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:38.753254 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 798/1578] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:14:38.753967 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3e63a5-86d1-48c3-9cbb-567ca10b0295 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:38.764200 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3e63a5-86d1-48c3-9cbb-567ca10b0295 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:38.769281 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aee0c2-3adb-44e0-b4bf-2673c27d5b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:38.769932 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aee0c2-3adb-44e0-b4bf-2673c27d5b1b admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:38.770216 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aee0c2-3adb-44e0-b4bf-2673c27d5b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:38.770493 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aee0c2-3adb-44e0-b4bf-2673c27d5b1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:38.771240 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aee0c2-3adb-44e0-b4bf-2673c27d5b1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:38.772810 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aee0c2-3adb-44e0-b4bf-2673c27d5b1b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:38.776579 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 781/1579] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:14:38 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:14:38.776579 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e0e32-4b05-4bfa-bfd2-e2efaec0c205 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:38.786255 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 799/1580] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:14:38.786255 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aee0c2-3adb-44e0-b4bf-2673c27d5b1b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:38.793925 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aee0c2-3adb-44e0-b4bf-2673c27d5b1b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:38.837072 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e0e32-4b05-4bfa-bfd2-e2efaec0c205 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:38.841738 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d6ebf6-3fc1-4efc-88dc-f55b64c1f83c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:38.842393 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6ebf6-3fc1-4efc-88dc-f55b64c1f83c admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:38.842963 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6ebf6-3fc1-4efc-88dc-f55b64c1f83c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:38.843245 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d6ebf6-3fc1-4efc-88dc-f55b64c1f83c admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:38.843931 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d6ebf6-3fc1-4efc-88dc-f55b64c1f83c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:38.845846 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d6ebf6-3fc1-4efc-88dc-f55b64c1f83c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:38.849361 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c768ece2-d8fa-41f2-9a72-abcb3aa9e831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:38.850254 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c768ece2-d8fa-41f2-9a72-abcb3aa9e831 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:38.850532 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c768ece2-d8fa-41f2-9a72-abcb3aa9e831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:38.850833 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c768ece2-d8fa-41f2-9a72-abcb3aa9e831 admin admin] PATH_INFO: `/v3/projects/d624b45721514810b49fbbd38c7afb5c/users/92e368db026e4a8f8f4730710471868b/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:38.866877 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c768ece2-d8fa-41f2-9a72-abcb3aa9e831 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d624b45721514810b49fbbd38c7afb5c, user_id=92e368db026e4a8f8f4730710471868b, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:38.867963 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: WARNING py.warnings [None req-c768ece2-d8fa-41f2-9a72-abcb3aa9e831 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 15:14:38.867963 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: warnings.warn(msg) May 27 15:14:38.867963 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: May 27 15:14:38.869929 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c768ece2-d8fa-41f2-9a72-abcb3aa9e831 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:38.872510 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 782/1581] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:14:38 2022] POST /identity/v3/projects => generated 371 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:14:38.873231 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d6ebf6-3fc1-4efc-88dc-f55b64c1f83c admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:38.887242 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 800/1582] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:14:38 2022] PUT /identity/v3/projects/d624b45721514810b49fbbd38c7afb5c/users/92e368db026e4a8f8f4730710471868b/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:38.891387 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c768ece2-d8fa-41f2-9a72-abcb3aa9e831 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:38.902258 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:INSERT=9,keystone:SELECT=91 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:38.953188 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c768ece2-d8fa-41f2-9a72-abcb3aa9e831 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:38.954500 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d6ebf6-3fc1-4efc-88dc-f55b64c1f83c admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:38.957874 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60cf2f9-ef7c-433f-a99a-f66947d911af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:38.958515 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60cf2f9-ef7c-433f-a99a-f66947d911af admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:38.958892 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60cf2f9-ef7c-433f-a99a-f66947d911af admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:38.959177 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60cf2f9-ef7c-433f-a99a-f66947d911af admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:38.959957 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60cf2f9-ef7c-433f-a99a-f66947d911af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:38.960814 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cf2bef-9250-4207-9c6d-5d9adaa3a6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:38.961476 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf2bef-9250-4207-9c6d-5d9adaa3a6cd admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:38.961586 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60cf2f9-ef7c-433f-a99a-f66947d911af admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:38.961761 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf2bef-9250-4207-9c6d-5d9adaa3a6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:38.962332 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cf2bef-9250-4207-9c6d-5d9adaa3a6cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:38.963114 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cf2bef-9250-4207-9c6d-5d9adaa3a6cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:38.964696 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cf2bef-9250-4207-9c6d-5d9adaa3a6cd admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:38.966410 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:38.975733 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 801/1583] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:14:38.975733 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60cf2f9-ef7c-433f-a99a-f66947d911af admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:38.978972 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 783/1584] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:14:38.978972 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cf2bef-9250-4207-9c6d-5d9adaa3a6cd admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:38.989559 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60cf2f9-ef7c-433f-a99a-f66947d911af admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:38.989779 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cf2bef-9250-4207-9c6d-5d9adaa3a6cd admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:38.994642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51f612-75c1-4438-939f-a7a210e13b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:38.995337 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51f612-75c1-4438-939f-a7a210e13b2c admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:38.995459 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb682c98-0be0-4626-b2f9-8efed6bf46cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:38.995622 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51f612-75c1-4438-939f-a7a210e13b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:38.995905 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd51f612-75c1-4438-939f-a7a210e13b2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:38.996293 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb682c98-0be0-4626-b2f9-8efed6bf46cb admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:38.997282 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb682c98-0be0-4626-b2f9-8efed6bf46cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:38.997453 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd51f612-75c1-4438-939f-a7a210e13b2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:38.997696 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb682c98-0be0-4626-b2f9-8efed6bf46cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:38.998489 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb682c98-0be0-4626-b2f9-8efed6bf46cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:38.999162 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd51f612-75c1-4438-939f-a7a210e13b2c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:39.000176 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb682c98-0be0-4626-b2f9-8efed6bf46cb admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:39.009463 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 802/1585] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:14:39.010302 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51f612-75c1-4438-939f-a7a210e13b2c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:39.010418 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 784/1586] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:14:39.011261 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb682c98-0be0-4626-b2f9-8efed6bf46cb admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:39.020903 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb682c98-0be0-4626-b2f9-8efed6bf46cb admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:39.021627 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd51f612-75c1-4438-939f-a7a210e13b2c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:39.025701 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c312d35f-c786-4258-8212-bc9b494a8bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:39.026551 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312d35f-c786-4258-8212-bc9b494a8bf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:39.026667 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312d35f-c786-4258-8212-bc9b494a8bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:39.026759 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3383a3f2-f109-401a-aace-ebda02b31dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:39.026983 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c312d35f-c786-4258-8212-bc9b494a8bf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:39.027380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3383a3f2-f109-401a-aace-ebda02b31dbf admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:39.027663 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c312d35f-c786-4258-8212-bc9b494a8bf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:39.027878 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3383a3f2-f109-401a-aace-ebda02b31dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:39.028097 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3383a3f2-f109-401a-aace-ebda02b31dbf admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:39.028882 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3383a3f2-f109-401a-aace-ebda02b31dbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:39.029475 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c312d35f-c786-4258-8212-bc9b494a8bf4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:39.030789 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3383a3f2-f109-401a-aace-ebda02b31dbf admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:39.040149 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 785/1587] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:14:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:14:39.040936 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c312d35f-c786-4258-8212-bc9b494a8bf4 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:39.052202 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 803/1588] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:14:39 2022] POST /identity/v3/users => generated 293 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:14:39.053545 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e037a382-5b98-4322-928a-21003b1e66f4 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:39.053839 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e037a382-5b98-4322-928a-21003b1e66f4 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:39.054009 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c312d35f-c786-4258-8212-bc9b494a8bf4 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:39.054102 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e037a382-5b98-4322-928a-21003b1e66f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:39.059973 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477e3da5-759d-4a14-926e-f2e547fb6dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:39.060933 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e3da5-759d-4a14-926e-f2e547fb6dac admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:39.061152 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e3da5-759d-4a14-926e-f2e547fb6dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:39.061492 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477e3da5-759d-4a14-926e-f2e547fb6dac admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:39.062189 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477e3da5-759d-4a14-926e-f2e547fb6dac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:39.064160 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477e3da5-759d-4a14-926e-f2e547fb6dac admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:39.073167 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e037a382-5b98-4322-928a-21003b1e66f4 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:39.074370 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e037a382-5b98-4322-928a-21003b1e66f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:39.082953 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 804/1589] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:14:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:39.083941 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e037a382-5b98-4322-928a-21003b1e66f4 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:39.088298 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 786/1590] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:14:39 2022] POST /identity/v3/users => generated 286 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:14:39.089247 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477e3da5-759d-4a14-926e-f2e547fb6dac admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:39.093833 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e037a382-5b98-4322-928a-21003b1e66f4 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:39.098471 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f93db53-63e7-4d57-a524-eae5801f4a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:39.099165 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f93db53-63e7-4d57-a524-eae5801f4a12 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:39.099457 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f93db53-63e7-4d57-a524-eae5801f4a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:39.099733 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f93db53-63e7-4d57-a524-eae5801f4a12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:39.100458 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f93db53-63e7-4d57-a524-eae5801f4a12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:39.101931 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477e3da5-759d-4a14-926e-f2e547fb6dac admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:39.102066 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f93db53-63e7-4d57-a524-eae5801f4a12 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:39.107179 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073cf94b-e101-4034-bc7d-28a44fb16285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:39.107839 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073cf94b-e101-4034-bc7d-28a44fb16285 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:39.108120 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073cf94b-e101-4034-bc7d-28a44fb16285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:39.108542 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073cf94b-e101-4034-bc7d-28a44fb16285 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:39.109333 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073cf94b-e101-4034-bc7d-28a44fb16285 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:39.110952 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073cf94b-e101-4034-bc7d-28a44fb16285 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:39.115526 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 805/1591] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:14:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:14:39.116197 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f93db53-63e7-4d57-a524-eae5801f4a12 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:39.125523 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 787/1592] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:14:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:14:39.126369 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073cf94b-e101-4034-bc7d-28a44fb16285 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:39.127725 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f93db53-63e7-4d57-a524-eae5801f4a12 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:39.132598 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c603be-71d8-47ff-87b7-af9cf4d69317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:39.133493 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c603be-71d8-47ff-87b7-af9cf4d69317 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:39.133778 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c603be-71d8-47ff-87b7-af9cf4d69317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:39.134056 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c603be-71d8-47ff-87b7-af9cf4d69317 admin admin] PATH_INFO: `/v3/projects/c5744215ba8d4f439159f9532f7d88b3/users/f0f600b3b58b43a0829c7430bb3eaf4d/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:39.138776 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073cf94b-e101-4034-bc7d-28a44fb16285 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:39.143761 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424d5e7d-6add-4a5b-95aa-c7f335e13d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:39.144418 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d5e7d-6add-4a5b-95aa-c7f335e13d70 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:39.144702 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d5e7d-6add-4a5b-95aa-c7f335e13d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:39.144982 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d5e7d-6add-4a5b-95aa-c7f335e13d70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:39.145808 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424d5e7d-6add-4a5b-95aa-c7f335e13d70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:39.150281 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424d5e7d-6add-4a5b-95aa-c7f335e13d70 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:39.151824 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c603be-71d8-47ff-87b7-af9cf4d69317 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5744215ba8d4f439159f9532f7d88b3, user_id=f0f600b3b58b43a0829c7430bb3eaf4d, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:39.155644 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c603be-71d8-47ff-87b7-af9cf4d69317 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:39.160039 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 788/1593] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:14:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:14:39.160892 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424d5e7d-6add-4a5b-95aa-c7f335e13d70 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:39.171258 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424d5e7d-6add-4a5b-95aa-c7f335e13d70 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:39.171443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 806/1594] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:14:39 2022] PUT /identity/v3/projects/c5744215ba8d4f439159f9532f7d88b3/users/f0f600b3b58b43a0829c7430bb3eaf4d/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:39.172186 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c603be-71d8-47ff-87b7-af9cf4d69317 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:39.233846 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cf475b-3387-414f-a11d-9f9d3505f323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:39.234731 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf475b-3387-414f-a11d-9f9d3505f323 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:39.235040 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf475b-3387-414f-a11d-9f9d3505f323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:39.235315 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf475b-3387-414f-a11d-9f9d3505f323 admin admin] PATH_INFO: `/v3/projects/0bf15a964f8048b48bc794051cd56add/users/387048f42b634693a7e827a8d88ef89b/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:39.238227 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c603be-71d8-47ff-87b7-af9cf4d69317 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:39.242893 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e9bae-5d6f-4502-ada6-a194c13eceea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:39.243583 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e9bae-5d6f-4502-ada6-a194c13eceea admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:39.243868 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e9bae-5d6f-4502-ada6-a194c13eceea admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:39.244267 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e9bae-5d6f-4502-ada6-a194c13eceea admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:39.245207 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9e9bae-5d6f-4502-ada6-a194c13eceea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:39.247175 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9e9bae-5d6f-4502-ada6-a194c13eceea admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:39.249904 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cf475b-3387-414f-a11d-9f9d3505f323 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bf15a964f8048b48bc794051cd56add, user_id=387048f42b634693a7e827a8d88ef89b, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:39.252567 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cf475b-3387-414f-a11d-9f9d3505f323 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:39.260761 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 807/1595] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:14:39 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:14:39.261683 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e9bae-5d6f-4502-ada6-a194c13eceea admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:39.270241 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 789/1596] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:14:39 2022] PUT /identity/v3/projects/0bf15a964f8048b48bc794051cd56add/users/387048f42b634693a7e827a8d88ef89b/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:39.271325 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cf475b-3387-414f-a11d-9f9d3505f323 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:39.275025 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e9bae-5d6f-4502-ada6-a194c13eceea admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:39.329412 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cf475b-3387-414f-a11d-9f9d3505f323 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:39.333172 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d822c4-21da-4f76-9e3d-abdb993b99d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:39.333851 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d822c4-21da-4f76-9e3d-abdb993b99d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:39.334138 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d822c4-21da-4f76-9e3d-abdb993b99d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:39.334421 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d822c4-21da-4f76-9e3d-abdb993b99d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:39.369434 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:39.402286 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d822c4-21da-4f76-9e3d-abdb993b99d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:39.404297 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d822c4-21da-4f76-9e3d-abdb993b99d8 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:39.414768 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 790/1597] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:14:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:14:39.415659 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d822c4-21da-4f76-9e3d-abdb993b99d8 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:39.426881 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d822c4-21da-4f76-9e3d-abdb993b99d8 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:39.431838 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea79337-2c00-484f-93b0-ed7df627f526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:39.432743 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea79337-2c00-484f-93b0-ed7df627f526 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:39.433039 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea79337-2c00-484f-93b0-ed7df627f526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:39.433318 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea79337-2c00-484f-93b0-ed7df627f526 admin admin] PATH_INFO: `/v3/projects/cce3ede867a04744ae536cb737e14639/users/3d99e26b3cd04bba98f43712e0ae17e5/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:39.441116 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:39.456657 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea79337-2c00-484f-93b0-ed7df627f526 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cce3ede867a04744ae536cb737e14639, user_id=3d99e26b3cd04bba98f43712e0ae17e5, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:39.459729 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea79337-2c00-484f-93b0-ed7df627f526 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:39.476609 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a07b8fe-3941-492b-97c3-8111cbe7534c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:39.477565 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07b8fe-3941-492b-97c3-8111cbe7534c admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:39.478062 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07b8fe-3941-492b-97c3-8111cbe7534c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:39.478512 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a07b8fe-3941-492b-97c3-8111cbe7534c admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:39.478697 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 791/1598] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:14:39 2022] PUT /identity/v3/projects/cce3ede867a04744ae536cb737e14639/users/3d99e26b3cd04bba98f43712e0ae17e5/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:39.479571 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea79337-2c00-484f-93b0-ed7df627f526 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:39.479921 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a07b8fe-3941-492b-97c3-8111cbe7534c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:39.482367 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a07b8fe-3941-492b-97c3-8111cbe7534c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:39.494225 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 808/1599] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:14:39 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:14:39.498410 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a07b8fe-3941-492b-97c3-8111cbe7534c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:39.509590 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a07b8fe-3941-492b-97c3-8111cbe7534c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:39.554605 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea79337-2c00-484f-93b0-ed7df627f526 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:39.560539 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1361f8e-c252-4636-88ac-a130aaf7ba30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:39.561673 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1361f8e-c252-4636-88ac-a130aaf7ba30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:39.561919 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1361f8e-c252-4636-88ac-a130aaf7ba30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:39.562223 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1361f8e-c252-4636-88ac-a130aaf7ba30 admin admin] PATH_INFO: `/v3/projects/d3a001740e9e476c9cee17b42c1c7816/users/37f29ccc2165414e91f51a38d2d918dd/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:39.575418 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc3c16c-1408-4dd5-9521-c309ddd8df11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:39.576744 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3c16c-1408-4dd5-9521-c309ddd8df11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:39.577177 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3c16c-1408-4dd5-9521-c309ddd8df11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:39.577748 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3c16c-1408-4dd5-9521-c309ddd8df11 admin admin] PATH_INFO: `/v3/projects/f20b51981801495aacd31a56e87d0e3c/users/5a990918de6d409a949616da90de6a5e/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:39.578934 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1361f8e-c252-4636-88ac-a130aaf7ba30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3a001740e9e476c9cee17b42c1c7816, user_id=37f29ccc2165414e91f51a38d2d918dd, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:39.582491 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1361f8e-c252-4636-88ac-a130aaf7ba30 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:39.596471 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc3c16c-1408-4dd5-9521-c309ddd8df11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f20b51981801495aacd31a56e87d0e3c, user_id=5a990918de6d409a949616da90de6a5e, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:39.596948 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 792/1600] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:14:39 2022] PUT /identity/v3/projects/d3a001740e9e476c9cee17b42c1c7816/users/37f29ccc2165414e91f51a38d2d918dd/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:39.598250 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf2a01-2525-45fb-9080-07a7440388f8 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:39.598535 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf2a01-2525-45fb-9080-07a7440388f8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:39.598811 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabf2a01-2525-45fb-9080-07a7440388f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:39.600488 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc3c16c-1408-4dd5-9521-c309ddd8df11 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:39.626613 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 809/1601] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:14:39 2022] PUT /identity/v3/projects/f20b51981801495aacd31a56e87d0e3c/users/5a990918de6d409a949616da90de6a5e/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:39.628299 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc3c16c-1408-4dd5-9521-c309ddd8df11 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:39.632220 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-cabf2a01-2525-45fb-9080-07a7440388f8 None None] MFA Rules not processed for user `f0f600b3b58b43a0829c7430bb3eaf4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:39.633423 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cabf2a01-2525-45fb-9080-07a7440388f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:39.699387 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc3c16c-1408-4dd5-9521-c309ddd8df11 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:39.705820 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7d6326-c7ea-4de3-8cab-b426529d74c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:39.706021 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 793/1602] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:14:39 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:39.707247 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d6326-c7ea-4de3-8cab-b426529d74c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:39.707387 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caabd40-35aa-4967-9318-1d297d089072 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:39.707766 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caabd40-35aa-4967-9318-1d297d089072 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:39.708010 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d6326-c7ea-4de3-8cab-b426529d74c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:39.708139 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caabd40-35aa-4967-9318-1d297d089072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:39.708555 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7d6326-c7ea-4de3-8cab-b426529d74c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:39.709425 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7d6326-c7ea-4de3-8cab-b426529d74c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:39.712092 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7d6326-c7ea-4de3-8cab-b426529d74c0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:39.723635 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 810/1603] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:14:39 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:14:39.728428 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cef2d4-d4bb-45d8-8cad-aced32f0c1c3 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:39.728577 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cef2d4-d4bb-45d8-8cad-aced32f0c1c3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:39.728831 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cef2d4-d4bb-45d8-8cad-aced32f0c1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:39.732517 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4caabd40-35aa-4967-9318-1d297d089072 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:39.733756 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4caabd40-35aa-4967-9318-1d297d089072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:39.742075 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 794/1604] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:14:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:39.743058 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caabd40-35aa-4967-9318-1d297d089072 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:39.751652 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-06cef2d4-d4bb-45d8-8cad-aced32f0c1c3 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:39.752553 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-06cef2d4-d4bb-45d8-8cad-aced32f0c1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:39.764163 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 811/1605] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:14:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:39.764163 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbdf47a-fe79-43ea-8520-c85bf992c820 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:39.764163 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbdf47a-fe79-43ea-8520-c85bf992c820 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:39.764163 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbdf47a-fe79-43ea-8520-c85bf992c820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:39.769437 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caabd40-35aa-4967-9318-1d297d089072 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:39.776419 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c4228a-28e0-4d78-a404-ec4a60a5cbf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:39.777201 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4228a-28e0-4d78-a404-ec4a60a5cbf3 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:39.777999 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4228a-28e0-4d78-a404-ec4a60a5cbf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:39.778884 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c4228a-28e0-4d78-a404-ec4a60a5cbf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:39.779426 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c4228a-28e0-4d78-a404-ec4a60a5cbf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:39.781110 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c4228a-28e0-4d78-a404-ec4a60a5cbf3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:39.783947 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:39.796494 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-edbdf47a-fe79-43ea-8520-c85bf992c820 None None] MFA Rules not processed for user `37f29ccc2165414e91f51a38d2d918dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:39.797058 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 795/1606] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:14:39 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:14:39.797995 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-edbdf47a-fe79-43ea-8520-c85bf992c820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:39.798501 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ce975-1b90-4a89-8377-dd3f7ee9d6fb None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:39.799203 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ce975-1b90-4a89-8377-dd3f7ee9d6fb None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:39.800071 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ce975-1b90-4a89-8377-dd3f7ee9d6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:39.818481 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-255ce975-1b90-4a89-8377-dd3f7ee9d6fb None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:39.819842 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-255ce975-1b90-4a89-8377-dd3f7ee9d6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:39.831583 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 796/1607] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:14:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:39.831583 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255ce975-1b90-4a89-8377-dd3f7ee9d6fb None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:39.835800 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:39.870335 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 812/1608] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:14:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:39.871267 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbdf47a-fe79-43ea-8520-c85bf992c820 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:39.884044 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbdf47a-fe79-43ea-8520-c85bf992c820 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:39.889845 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0a66f3-0fc2-4c9b-9cb5-3a628933931d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:39.891060 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a66f3-0fc2-4c9b-9cb5-3a628933931d admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:39.891350 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a66f3-0fc2-4c9b-9cb5-3a628933931d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:39.891808 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a66f3-0fc2-4c9b-9cb5-3a628933931d admin admin] PATH_INFO: `/v3/projects/f0964b39386343c5997a50981be53341/users/c899cd80a3dc4f15be32d3f26dce4b1f/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:39.913854 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0a66f3-0fc2-4c9b-9cb5-3a628933931d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0964b39386343c5997a50981be53341, user_id=c899cd80a3dc4f15be32d3f26dce4b1f, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:39.915626 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255ce975-1b90-4a89-8377-dd3f7ee9d6fb None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:39.917147 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0a66f3-0fc2-4c9b-9cb5-3a628933931d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:39.924898 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71959593-f59d-4278-86e7-3222a51910b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:39.924898 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71959593-f59d-4278-86e7-3222a51910b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:39.924898 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71959593-f59d-4278-86e7-3222a51910b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:39.924898 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71959593-f59d-4278-86e7-3222a51910b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:39.934181 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71959593-f59d-4278-86e7-3222a51910b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:39.936840 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71959593-f59d-4278-86e7-3222a51910b7 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:39.938917 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 813/1609] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:14:39 2022] PUT /identity/v3/projects/f0964b39386343c5997a50981be53341/users/c899cd80a3dc4f15be32d3f26dce4b1f/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:39.940287 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0a66f3-0fc2-4c9b-9cb5-3a628933931d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:39.950521 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 797/1610] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:14:39 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:14:39.951423 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71959593-f59d-4278-86e7-3222a51910b7 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:40.013211 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71959593-f59d-4278-86e7-3222a51910b7 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:40.037236 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0a66f3-0fc2-4c9b-9cb5-3a628933931d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:40.042500 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb9ef29-32c1-4379-b687-e0921407187a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:40.044443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9ef29-32c1-4379-b687-e0921407187a service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:40.044887 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9ef29-32c1-4379-b687-e0921407187a service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:40.045858 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9ef29-32c1-4379-b687-e0921407187a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:40.088191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fce67e-f45f-45c3-ae9f-42bf998e4f5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:40.088928 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fce67e-f45f-45c3-ae9f-42bf998e4f5d service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:40.089220 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fce67e-f45f-45c3-ae9f-42bf998e4f5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:40.089628 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fce67e-f45f-45c3-ae9f-42bf998e4f5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:40.160882 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb9ef29-32c1-4379-b687-e0921407187a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:40.162424 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb9ef29-32c1-4379-b687-e0921407187a service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:40.175215 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 814/1611] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:14:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:14:40.176393 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb9ef29-32c1-4379-b687-e0921407187a service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:40.187737 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb9ef29-32c1-4379-b687-e0921407187a service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:40.193940 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0923d3e2-c208-4dc7-96b2-7ed5768fa2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:40.194994 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0923d3e2-c208-4dc7-96b2-7ed5768fa2ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:40.195284 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0923d3e2-c208-4dc7-96b2-7ed5768fa2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:40.195608 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0923d3e2-c208-4dc7-96b2-7ed5768fa2ff admin admin] PATH_INFO: `/v3/projects/efb8b11b65d04ce5a16afcc8ab579931/users/5148faabb9414eeca471f00591eb167d/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:40.201004 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fce67e-f45f-45c3-ae9f-42bf998e4f5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:40.202516 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fce67e-f45f-45c3-ae9f-42bf998e4f5d service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:40.208765 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:40.214886 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 798/1612] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:14:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:14:40.215078 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0923d3e2-c208-4dc7-96b2-7ed5768fa2ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=efb8b11b65d04ce5a16afcc8ab579931, user_id=5148faabb9414eeca471f00591eb167d, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:40.216519 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fce67e-f45f-45c3-ae9f-42bf998e4f5d service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:40.219030 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0923d3e2-c208-4dc7-96b2-7ed5768fa2ff admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:40.228617 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fce67e-f45f-45c3-ae9f-42bf998e4f5d service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:40.246950 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 815/1613] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:14:40 2022] PUT /identity/v3/projects/efb8b11b65d04ce5a16afcc8ab579931/users/5148faabb9414eeca471f00591eb167d/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:40.246950 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86c938-c525-4913-a284-2b398bb59c28 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:40.246950 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86c938-c525-4913-a284-2b398bb59c28 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:40.246950 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86c938-c525-4913-a284-2b398bb59c28 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:40.246950 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 816/1614] 10.222.0.50 () {54 vars in 961 bytes} [Fri May 27 15:14:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:14:40.246950 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f3ff2-c6bc-46b0-b6b6-88d4ae769dc9 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:40.247606 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f3ff2-c6bc-46b0-b6b6-88d4ae769dc9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:40.247606 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f3ff2-c6bc-46b0-b6b6-88d4ae769dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:40.283799 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-637f3ff2-c6bc-46b0-b6b6-88d4ae769dc9 None None] MFA Rules not processed for user `c899cd80a3dc4f15be32d3f26dce4b1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:40.285414 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-637f3ff2-c6bc-46b0-b6b6-88d4ae769dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:40.311088 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2383ad66-f717-4b5f-85d5-c22ef3318a0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:40.312126 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2383ad66-f717-4b5f-85d5-c22ef3318a0e service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:40.312569 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2383ad66-f717-4b5f-85d5-c22ef3318a0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:40.313033 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2383ad66-f717-4b5f-85d5-c22ef3318a0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:40.323316 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:40.390393 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 817/1615] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:14:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:40.392998 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c01f7-0cf6-4488-9a1e-16b22ade07f7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:40.393485 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c01f7-0cf6-4488-9a1e-16b22ade07f7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:40.393682 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01c01f7-0cf6-4488-9a1e-16b22ade07f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:40.397054 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2383ad66-f717-4b5f-85d5-c22ef3318a0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:40.401440 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2383ad66-f717-4b5f-85d5-c22ef3318a0e service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:40.414072 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 799/1616] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:14:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:14:40.416262 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e5aa7-aac4-42b7-a8ef-9926db0457c8 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:40.416661 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e5aa7-aac4-42b7-a8ef-9926db0457c8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:40.417044 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e5aa7-aac4-42b7-a8ef-9926db0457c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:40.440234 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b01c01f7-0cf6-4488-9a1e-16b22ade07f7 None None] MFA Rules not processed for user `5148faabb9414eeca471f00591eb167d`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:40.442450 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b01c01f7-0cf6-4488-9a1e-16b22ade07f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:40.442703 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ad9e5aa7-aac4-42b7-a8ef-9926db0457c8 None None] MFA Rules not processed for user `451a9232f95e494683d81b6078a318d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:40.444391 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ad9e5aa7-aac4-42b7-a8ef-9926db0457c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:40.550582 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 818/1617] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:14:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:40.551637 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01c01f7-0cf6-4488-9a1e-16b22ade07f7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:40.573711 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01c01f7-0cf6-4488-9a1e-16b22ade07f7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:40.581131 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9294be-f63e-4f4c-bdf0-bb3e35d0640a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:40.581997 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9294be-f63e-4f4c-bdf0-bb3e35d0640a admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:40.582245 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9294be-f63e-4f4c-bdf0-bb3e35d0640a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:40.582531 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9294be-f63e-4f4c-bdf0-bb3e35d0640a admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:40.584144 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9294be-f63e-4f4c-bdf0-bb3e35d0640a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:40.586583 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9294be-f63e-4f4c-bdf0-bb3e35d0640a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:40.642720 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 800/1618] 10.222.0.50 () {58 vars in 1058 bytes} [Fri May 27 15:14:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:40.653170 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 819/1619] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:14:40 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:14:40.654842 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f1d41-e7f8-4778-883b-3f2bf9283a1d None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:40.655202 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f1d41-e7f8-4778-883b-3f2bf9283a1d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:40.655496 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1f1d41-e7f8-4778-883b-3f2bf9283a1d None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:40.658998 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6e2fc-e3a3-46fb-bde2-4165af4249f7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:40.659365 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6e2fc-e3a3-46fb-bde2-4165af4249f7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:40.660436 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6e2fc-e3a3-46fb-bde2-4165af4249f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:40.662677 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 801/1620] 10.222.0.50 () {54 vars in 961 bytes} [Fri May 27 15:14:40 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:14:40.663279 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f1d41-e7f8-4778-883b-3f2bf9283a1d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:40.711378 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-86a6e2fc-e3a3-46fb-bde2-4165af4249f7 None None] MFA Rules not processed for user `92e368db026e4a8f8f4730710471868b`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:40.713218 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-86a6e2fc-e3a3-46fb-bde2-4165af4249f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:40.785395 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:40.848261 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1f1d41-e7f8-4778-883b-3f2bf9283a1d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:40.858570 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0811ee7-bc2c-464f-b3ca-1fac3e0d3477 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '451a9232f95e494683d81b6078a318d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:40.859289 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0811ee7-bc2c-464f-b3ca-1fac3e0d3477 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:40.859574 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0811ee7-bc2c-464f-b3ca-1fac3e0d3477 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:40.859855 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0811ee7-bc2c-464f-b3ca-1fac3e0d3477 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:40.959098 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 820/1621] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:14:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:40.961859 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a6e2fc-e3a3-46fb-bde2-4165af4249f7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:40.987858 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0811ee7-bc2c-464f-b3ca-1fac3e0d3477 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:40.990632 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0811ee7-bc2c-464f-b3ca-1fac3e0d3477 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:41.005783 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 802/1622] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:14:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:14:41.007994 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e898d665-ea15-41a4-8b1a-773e1bfed242 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:41.008421 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e898d665-ea15-41a4-8b1a-773e1bfed242 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:41.008832 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e898d665-ea15-41a4-8b1a-773e1bfed242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:41.011639 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:41.039667 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e898d665-ea15-41a4-8b1a-773e1bfed242 None None] MFA Rules not processed for user `451a9232f95e494683d81b6078a318d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:41.041139 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e898d665-ea15-41a4-8b1a-773e1bfed242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:41.053563 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 803/1623] 10.222.0.50 () {58 vars in 1058 bytes} [Fri May 27 15:14:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:41.055227 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e366ec4-db11-4b15-9141-d033312ff6d4 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:41.055681 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e366ec4-db11-4b15-9141-d033312ff6d4 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:41.056120 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e366ec4-db11-4b15-9141-d033312ff6d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:41.070157 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a6e2fc-e3a3-46fb-bde2-4165af4249f7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:41.076659 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6fe2cd-6a10-4a9f-8c02-056cbae1e909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:41.077411 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5e366ec4-db11-4b15-9141-d033312ff6d4 None None] MFA Rules not processed for user `92e368db026e4a8f8f4730710471868b`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:41.078917 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6fe2cd-6a10-4a9f-8c02-056cbae1e909 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:41.079319 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5e366ec4-db11-4b15-9141-d033312ff6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:41.079988 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6fe2cd-6a10-4a9f-8c02-056cbae1e909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:41.080372 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6fe2cd-6a10-4a9f-8c02-056cbae1e909 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:41.081179 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6fe2cd-6a10-4a9f-8c02-056cbae1e909 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:41.083654 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6fe2cd-6a10-4a9f-8c02-056cbae1e909 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:41.088770 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 804/1624] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:14:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:41.090056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e366ec4-db11-4b15-9141-d033312ff6d4 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:41.109046 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e366ec4-db11-4b15-9141-d033312ff6d4 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:41.125547 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2652e8-00aa-4871-acaf-43ac7c7b87af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '451a9232f95e494683d81b6078a318d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:41.126554 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2652e8-00aa-4871-acaf-43ac7c7b87af service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:41.127270 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2652e8-00aa-4871-acaf-43ac7c7b87af service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:41.127454 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2652e8-00aa-4871-acaf-43ac7c7b87af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:41.129018 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 821/1625] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:14:40 2022] POST /identity/v3/users => generated 294 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:14:41.132324 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6fe2cd-6a10-4a9f-8c02-056cbae1e909 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:41.146040 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6fe2cd-6a10-4a9f-8c02-056cbae1e909 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:41.162590 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0196c9c7-7ce2-426d-b75a-9e880d4d06c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:41.163439 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196c9c7-7ce2-426d-b75a-9e880d4d06c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:41.163830 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196c9c7-7ce2-426d-b75a-9e880d4d06c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:41.163997 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196c9c7-7ce2-426d-b75a-9e880d4d06c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:41.166429 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0196c9c7-7ce2-426d-b75a-9e880d4d06c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:41.169412 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0196c9c7-7ce2-426d-b75a-9e880d4d06c6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:41.186200 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 822/1626] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:14:41.187323 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0196c9c7-7ce2-426d-b75a-9e880d4d06c6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:41.198863 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0196c9c7-7ce2-426d-b75a-9e880d4d06c6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:41.205356 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a51a3ac-e329-4cb9-9772-5e3a10fd3b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:41.206566 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51a3ac-e329-4cb9-9772-5e3a10fd3b24 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:41.206707 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51a3ac-e329-4cb9-9772-5e3a10fd3b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:41.207050 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51a3ac-e329-4cb9-9772-5e3a10fd3b24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:41.207948 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a51a3ac-e329-4cb9-9772-5e3a10fd3b24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:41.210104 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a51a3ac-e329-4cb9-9772-5e3a10fd3b24 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:41.218363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2652e8-00aa-4871-acaf-43ac7c7b87af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:41.221293 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2652e8-00aa-4871-acaf-43ac7c7b87af service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:41.229175 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 823/1627] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:14:41.229615 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a51a3ac-e329-4cb9-9772-5e3a10fd3b24 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:41.245334 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a51a3ac-e329-4cb9-9772-5e3a10fd3b24 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:41.248681 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 805/1628] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:14:41 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:14:41.264686 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef62f16-40a4-4ab6-a352-a53f29c61c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:41.272292 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef62f16-40a4-4ab6-a352-a53f29c61c05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:41.272518 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef62f16-40a4-4ab6-a352-a53f29c61c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:41.272813 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef62f16-40a4-4ab6-a352-a53f29c61c05 admin admin] PATH_INFO: `/v3/projects/d972ac5aa1094b618977c30771245f6e/users/9322200d30be4cad976ebc35009d0572/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:41.299645 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef62f16-40a4-4ab6-a352-a53f29c61c05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d972ac5aa1094b618977c30771245f6e, user_id=9322200d30be4cad976ebc35009d0572, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:41.302467 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef62f16-40a4-4ab6-a352-a53f29c61c05 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:41.335820 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 824/1629] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:14:41 2022] PUT /identity/v3/projects/d972ac5aa1094b618977c30771245f6e/users/9322200d30be4cad976ebc35009d0572/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:41.349211 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d65c7-6849-4f1e-be54-2a187a6910bd None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:41.349655 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d65c7-6849-4f1e-be54-2a187a6910bd None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:41.350119 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3d65c7-6849-4f1e-be54-2a187a6910bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:41.394635 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4456b495-9f77-4b48-9e00-1686dae2e02f None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:41.394965 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4456b495-9f77-4b48-9e00-1686dae2e02f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:41.395237 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4456b495-9f77-4b48-9e00-1686dae2e02f None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:41.397075 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 825/1630] 10.222.0.50 () {54 vars in 912 bytes} [Fri May 27 15:14:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:14:41.408563 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-cd3d65c7-6849-4f1e-be54-2a187a6910bd None None] MFA Rules not processed for user `9322200d30be4cad976ebc35009d0572`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:41.409976 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cd3d65c7-6849-4f1e-be54-2a187a6910bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:41.421771 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd60a1-83ed-4ab2-a284-961990cbff57 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:41.422575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd60a1-83ed-4ab2-a284-961990cbff57 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:41.423115 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd60a1-83ed-4ab2-a284-961990cbff57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:41.457944 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-accd60a1-83ed-4ab2-a284-961990cbff57 None None] MFA Rules not processed for user `451a9232f95e494683d81b6078a318d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:41.459539 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-accd60a1-83ed-4ab2-a284-961990cbff57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:41.491338 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:41.511205 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 806/1631] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:14:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:41.513439 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3d65c7-6849-4f1e-be54-2a187a6910bd None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:41.653442 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 826/1632] 10.222.0.50 () {58 vars in 1009 bytes} [Fri May 27 15:14:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:14:41.670886 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accd60a1-83ed-4ab2-a284-961990cbff57 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:41.725461 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3d65c7-6849-4f1e-be54-2a187a6910bd None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:41.737903 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca8aceb-0692-4a19-8833-2f7c12de67f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:41.738609 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca8aceb-0692-4a19-8833-2f7c12de67f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:41.738983 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca8aceb-0692-4a19-8833-2f7c12de67f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:41.739276 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca8aceb-0692-4a19-8833-2f7c12de67f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:41.740042 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca8aceb-0692-4a19-8833-2f7c12de67f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:41.742173 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca8aceb-0692-4a19-8833-2f7c12de67f9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:41.781742 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 807/1633] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:14:41 2022] POST /identity/v3/projects => generated 385 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:14:41.783573 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb96770-d3ed-4adf-8fcc-3a16949f966d None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:41.783870 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb96770-d3ed-4adf-8fcc-3a16949f966d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:41.784527 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb96770-d3ed-4adf-8fcc-3a16949f966d None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:41.786423 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 808/1634] 10.222.0.50 () {54 vars in 912 bytes} [Fri May 27 15:14:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:14:41.787259 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb96770-d3ed-4adf-8fcc-3a16949f966d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:41.807007 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accd60a1-83ed-4ab2-a284-961990cbff57 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:41.828267 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:41.886727 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb96770-d3ed-4adf-8fcc-3a16949f966d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:41.894125 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0aa276-18a3-443d-ae41-036bc187eacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:41.894814 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0aa276-18a3-443d-ae41-036bc187eacb admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:41.895182 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0aa276-18a3-443d-ae41-036bc187eacb admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:41.895464 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0aa276-18a3-443d-ae41-036bc187eacb admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:41.896404 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0aa276-18a3-443d-ae41-036bc187eacb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:41.898687 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0aa276-18a3-443d-ae41-036bc187eacb admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:41.919898 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbba08c9-3fba-4a21-ae79-e124c0000711 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:41.921741 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba08c9-3fba-4a21-ae79-e124c0000711 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:41.929521 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba08c9-3fba-4a21-ae79-e124c0000711 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:41.929954 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba08c9-3fba-4a21-ae79-e124c0000711 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:41.932492 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 809/1635] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:14:41 2022] POST /identity/v3/users => generated 299 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:14:41.935197 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb8607-fbc6-472a-9f95-ca32b6654b6a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:41.935683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb8607-fbc6-472a-9f95-ca32b6654b6a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:41.936483 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeb8607-fbc6-472a-9f95-ca32b6654b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:41.958294 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ceeb8607-fbc6-472a-9f95-ca32b6654b6a None None] MFA Rules not processed for user `451a9232f95e494683d81b6078a318d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:41.959568 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ceeb8607-fbc6-472a-9f95-ca32b6654b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:42.031494 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbba08c9-3fba-4a21-ae79-e124c0000711 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:42.032467 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:42.033776 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbba08c9-3fba-4a21-ae79-e124c0000711 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:42.046221 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 827/1636] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:14:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:14:42.048681 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 810/1637] 10.222.0.50 () {58 vars in 1009 bytes} [Fri May 27 15:14:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:42.060994 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbba08c9-3fba-4a21-ae79-e124c0000711 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:42.062033 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeb8607-fbc6-472a-9f95-ca32b6654b6a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:42.079604 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeb8607-fbc6-472a-9f95-ca32b6654b6a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:42.084027 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbba08c9-3fba-4a21-ae79-e124c0000711 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:42.084188 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eea6b78-c143-4b84-8604-99ff59f9dc2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:42.084819 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea6b78-c143-4b84-8604-99ff59f9dc2e service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:42.085138 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea6b78-c143-4b84-8604-99ff59f9dc2e service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:42.085419 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eea6b78-c143-4b84-8604-99ff59f9dc2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:42.089875 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12983d21-2b41-48ce-87dc-b3c00da429ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:42.091742 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12983d21-2b41-48ce-87dc-b3c00da429ec admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:42.092252 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12983d21-2b41-48ce-87dc-b3c00da429ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:42.092984 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12983d21-2b41-48ce-87dc-b3c00da429ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:42.097439 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12983d21-2b41-48ce-87dc-b3c00da429ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:42.099659 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12983d21-2b41-48ce-87dc-b3c00da429ec admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:42.099994 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eea6b78-c143-4b84-8604-99ff59f9dc2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:42.102060 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eea6b78-c143-4b84-8604-99ff59f9dc2e service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:42.116934 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 828/1638] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:14:42.122159 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12983d21-2b41-48ce-87dc-b3c00da429ec admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:42.131152 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 811/1639] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:14:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:14:42.152023 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12983d21-2b41-48ce-87dc-b3c00da429ec admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:42.158752 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61506d3-c432-4856-87a2-a067a52bbf62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:42.160077 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61506d3-c432-4856-87a2-a067a52bbf62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:42.160453 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61506d3-c432-4856-87a2-a067a52bbf62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:42.160871 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61506d3-c432-4856-87a2-a067a52bbf62 admin admin] PATH_INFO: `/v3/projects/135d51d1a8be4ba593cc5eab07b74c68/users/096845fd015e44f68c14b740d386cc63/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:42.195667 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61506d3-c432-4856-87a2-a067a52bbf62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=135d51d1a8be4ba593cc5eab07b74c68, user_id=096845fd015e44f68c14b740d386cc63, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:42.203243 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61506d3-c432-4856-87a2-a067a52bbf62 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:42.225006 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 829/1640] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:14:42 2022] PUT /identity/v3/projects/135d51d1a8be4ba593cc5eab07b74c68/users/096845fd015e44f68c14b740d386cc63/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:42.229739 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eea6b78-c143-4b84-8604-99ff59f9dc2e service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:42.243535 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eea6b78-c143-4b84-8604-99ff59f9dc2e service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:42.356393 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371ee4d0-7609-4f2e-b9d9-78450f29f43b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:42.357351 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ee4d0-7609-4f2e-b9d9-78450f29f43b admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:42.357844 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ee4d0-7609-4f2e-b9d9-78450f29f43b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:42.358548 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ee4d0-7609-4f2e-b9d9-78450f29f43b admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:42.361001 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371ee4d0-7609-4f2e-b9d9-78450f29f43b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:42.364830 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371ee4d0-7609-4f2e-b9d9-78450f29f43b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:42.385112 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 812/1641] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:14:42.386520 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61506d3-c432-4856-87a2-a067a52bbf62 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:42.405210 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61506d3-c432-4856-87a2-a067a52bbf62 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:42.412581 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc9a191-7ec5-4874-9e23-a5fa964a86b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:42.414523 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9a191-7ec5-4874-9e23-a5fa964a86b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:42.415178 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9a191-7ec5-4874-9e23-a5fa964a86b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:42.416390 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc9a191-7ec5-4874-9e23-a5fa964a86b0 admin admin] PATH_INFO: `/v3/projects/135d51d1a8be4ba593cc5eab07b74c68/users/096845fd015e44f68c14b740d386cc63/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:42.421576 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc9a191-7ec5-4874-9e23-a5fa964a86b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=135d51d1a8be4ba593cc5eab07b74c68, user_id=096845fd015e44f68c14b740d386cc63, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:42.428565 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a27a75e-7614-4c25-b766-e70f031df913 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:42.428953 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a27a75e-7614-4c25-b766-e70f031df913 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:42.429336 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a27a75e-7614-4c25-b766-e70f031df913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:42.438158 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc9a191-7ec5-4874-9e23-a5fa964a86b0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:42.459924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 830/1642] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:14:42 2022] PUT /identity/v3/projects/135d51d1a8be4ba593cc5eab07b74c68/users/096845fd015e44f68c14b740d386cc63/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:14:42.472574 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a0782-e19f-40f1-9005-8fcd94c37a09 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:42.473555 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a0782-e19f-40f1-9005-8fcd94c37a09 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:42.473555 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3a0782-e19f-40f1-9005-8fcd94c37a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:42.533679 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8a3a0782-e19f-40f1-9005-8fcd94c37a09 None None] MFA Rules not processed for user `096845fd015e44f68c14b740d386cc63`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:42.535119 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8a3a0782-e19f-40f1-9005-8fcd94c37a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:42.553281 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-9a27a75e-7614-4c25-b766-e70f031df913 None None] MFA Rules not processed for user `3d99e26b3cd04bba98f43712e0ae17e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:42.555862 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9a27a75e-7614-4c25-b766-e70f031df913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:42.649269 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:42.681984 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 831/1643] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:14:42 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:14:42.716862 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 813/1644] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:14:42 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:14:42.721654 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3a0782-e19f-40f1-9005-8fcd94c37a09 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:42.832871 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3a0782-e19f-40f1-9005-8fcd94c37a09 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:42.838761 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a27a75e-7614-4c25-b766-e70f031df913 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:42.845319 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0cb3d7-38b6-4122-b379-b1b5e8daa9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:42.846237 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0cb3d7-38b6-4122-b379-b1b5e8daa9ee admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:42.848295 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0cb3d7-38b6-4122-b379-b1b5e8daa9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:42.848664 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0cb3d7-38b6-4122-b379-b1b5e8daa9ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:42.849544 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0cb3d7-38b6-4122-b379-b1b5e8daa9ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:42.853603 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0cb3d7-38b6-4122-b379-b1b5e8daa9ee admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:42.913076 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 832/1645] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:14:42 2022] POST /identity/v3/projects => generated 405 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:14:42.917066 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0cb3d7-38b6-4122-b379-b1b5e8daa9ee admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:43.067581 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a27a75e-7614-4c25-b766-e70f031df913 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:43.093744 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0cb3d7-38b6-4122-b379-b1b5e8daa9ee admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:43.100143 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b92dd49-e27a-4bc0-b30e-d37a6ab4a2e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '451a9232f95e494683d81b6078a318d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:43.101021 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b92dd49-e27a-4bc0-b30e-d37a6ab4a2e2 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:43.101474 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b92dd49-e27a-4bc0-b30e-d37a6ab4a2e2 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:43.101916 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b92dd49-e27a-4bc0-b30e-d37a6ab4a2e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:43.128980 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:43.181390 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b0f5c1-fe14-4125-a4d8-16fe6e5587eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:43.182056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b0f5c1-fe14-4125-a4d8-16fe6e5587eb service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:43.182336 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b0f5c1-fe14-4125-a4d8-16fe6e5587eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:43.182615 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b0f5c1-fe14-4125-a4d8-16fe6e5587eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:43.227693 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b92dd49-e27a-4bc0-b30e-d37a6ab4a2e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:43.229121 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b92dd49-e27a-4bc0-b30e-d37a6ab4a2e2 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:43.251560 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 833/1646] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:14:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:14:43.252398 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b92dd49-e27a-4bc0-b30e-d37a6ab4a2e2 service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:43.289344 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:43.324068 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b0f5c1-fe14-4125-a4d8-16fe6e5587eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:43.326032 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b0f5c1-fe14-4125-a4d8-16fe6e5587eb service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:43.351773 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 814/1647] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:14:42 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:14:43.389112 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b92dd49-e27a-4bc0-b30e-d37a6ab4a2e2 service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:43.396684 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ef69d2-6291-4137-ac52-f55eaa988786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:43.397001 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef69d2-6291-4137-ac52-f55eaa988786 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:43.397172 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef69d2-6291-4137-ac52-f55eaa988786 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:43.397505 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ef69d2-6291-4137-ac52-f55eaa988786 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:43.398373 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ef69d2-6291-4137-ac52-f55eaa988786 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:43.400493 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ef69d2-6291-4137-ac52-f55eaa988786 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:43.443427 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 834/1648] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:14:43 2022] POST /identity/v3/users => generated 309 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:14:43.447596 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b0f5c1-fe14-4125-a4d8-16fe6e5587eb service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:43.466997 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b0f5c1-fe14-4125-a4d8-16fe6e5587eb service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:43.474686 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005be5b4-36b2-459a-baa5-598822cc8085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:43.475772 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005be5b4-36b2-459a-baa5-598822cc8085 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:43.476395 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005be5b4-36b2-459a-baa5-598822cc8085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:43.476877 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005be5b4-36b2-459a-baa5-598822cc8085 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:43.477894 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005be5b4-36b2-459a-baa5-598822cc8085 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:43.480201 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005be5b4-36b2-459a-baa5-598822cc8085 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:43.493390 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 815/1649] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:14:43 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:14:43.498244 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ef69d2-6291-4137-ac52-f55eaa988786 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:43.510207 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ef69d2-6291-4137-ac52-f55eaa988786 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:43.536362 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c7e235-a12c-44f6-ac35-a7bf38acec26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:43.538611 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7e235-a12c-44f6-ac35-a7bf38acec26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:43.539141 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7e235-a12c-44f6-ac35-a7bf38acec26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:43.541325 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7e235-a12c-44f6-ac35-a7bf38acec26 admin admin] PATH_INFO: `/v3/projects/435f9c8ceeb248ac94af711caa2249dc/users/0692caa0f5ce47099cf5e549e6be70ac/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:43.563628 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c7e235-a12c-44f6-ac35-a7bf38acec26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=435f9c8ceeb248ac94af711caa2249dc, user_id=0692caa0f5ce47099cf5e549e6be70ac, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:43.567249 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c7e235-a12c-44f6-ac35-a7bf38acec26 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:43.600879 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 835/1650] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:14:43 2022] PUT /identity/v3/projects/435f9c8ceeb248ac94af711caa2249dc/users/0692caa0f5ce47099cf5e549e6be70ac/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:14:43.602318 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005be5b4-36b2-459a-baa5-598822cc8085 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:43.622710 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005be5b4-36b2-459a-baa5-598822cc8085 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:43.772661 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9fd485-3d20-49f3-923e-7a1c8a38b81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:43.773434 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9fd485-3d20-49f3-923e-7a1c8a38b81a admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:43.773838 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9fd485-3d20-49f3-923e-7a1c8a38b81a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:43.774395 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9fd485-3d20-49f3-923e-7a1c8a38b81a admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:43.775327 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9fd485-3d20-49f3-923e-7a1c8a38b81a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:43.780080 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9fd485-3d20-49f3-923e-7a1c8a38b81a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:43.817001 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 816/1651] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:14:43 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:14:43.819899 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c7e235-a12c-44f6-ac35-a7bf38acec26 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:43.845404 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c7e235-a12c-44f6-ac35-a7bf38acec26 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:43.857641 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a99aa2f-f1ed-48aa-b487-6e3be197dae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:43.858574 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a99aa2f-f1ed-48aa-b487-6e3be197dae6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:43.858874 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a99aa2f-f1ed-48aa-b487-6e3be197dae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:43.859167 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a99aa2f-f1ed-48aa-b487-6e3be197dae6 admin admin] PATH_INFO: `/v3/projects/435f9c8ceeb248ac94af711caa2249dc/users/0692caa0f5ce47099cf5e549e6be70ac/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:43.865867 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a99aa2f-f1ed-48aa-b487-6e3be197dae6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=435f9c8ceeb248ac94af711caa2249dc, user_id=0692caa0f5ce47099cf5e549e6be70ac, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:43.873924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a99aa2f-f1ed-48aa-b487-6e3be197dae6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:43.899856 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 836/1652] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:14:43 2022] PUT /identity/v3/projects/435f9c8ceeb248ac94af711caa2249dc/users/0692caa0f5ce47099cf5e549e6be70ac/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:43.958190 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e47327-5dd4-4ad5-b4b4-bde90a28fb8d None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:43.958792 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e47327-5dd4-4ad5-b4b4-bde90a28fb8d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:43.958792 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e47327-5dd4-4ad5-b4b4-bde90a28fb8d None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:43.961646 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 817/1653] 10.222.0.50 () {54 vars in 912 bytes} [Fri May 27 15:14:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:14:43.970778 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df559fb1-582a-4a72-866a-799ca040784f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:43.971128 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df559fb1-582a-4a72-866a-799ca040784f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:43.971365 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df559fb1-582a-4a72-866a-799ca040784f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:44.053432 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-df559fb1-582a-4a72-866a-799ca040784f None None] MFA Rules not processed for user `f874edc08438434189f1d14d79291487`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:44.060042 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-df559fb1-582a-4a72-866a-799ca040784f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:44.253063 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 837/1654] 10.222.0.50 () {58 vars in 1009 bytes} [Fri May 27 15:14:43 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:44.312961 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e47327-5dd4-4ad5-b4b4-bde90a28fb8d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:44.335912 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e47327-5dd4-4ad5-b4b4-bde90a28fb8d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:44.345243 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01c7ead-1251-4819-b51c-769061ca53e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:44.346132 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c7ead-1251-4819-b51c-769061ca53e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:44.348467 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c7ead-1251-4819-b51c-769061ca53e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:44.349182 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c7ead-1251-4819-b51c-769061ca53e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:44.385022 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01c7ead-1251-4819-b51c-769061ca53e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:44.394737 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01c7ead-1251-4819-b51c-769061ca53e6 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:44.412448 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 818/1655] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:14:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:14:44.485732 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df559fb1-582a-4a72-866a-799ca040784f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:44.677393 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df559fb1-582a-4a72-866a-799ca040784f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:44.683745 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e1cfde-91ba-41cd-b33f-9e3002189dbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:44.684625 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1cfde-91ba-41cd-b33f-9e3002189dbe service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:44.685155 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1cfde-91ba-41cd-b33f-9e3002189dbe service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:44.685613 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1cfde-91ba-41cd-b33f-9e3002189dbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:44.704389 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:44.793030 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e1cfde-91ba-41cd-b33f-9e3002189dbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:44.794529 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e1cfde-91ba-41cd-b33f-9e3002189dbe service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:44.807006 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 838/1656] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:14:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:14:45.430608 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139effc-0137-431b-b964-a6aac3e8f168 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:45.431396 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139effc-0137-431b-b964-a6aac3e8f168 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:45.431396 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d139effc-0137-431b-b964-a6aac3e8f168 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:45.437363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 819/1657] 10.222.0.50 () {54 vars in 960 bytes} [Fri May 27 15:14:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:14:45.450532 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50404881-6346-43d1-b2fb-72129eaa7c6a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:45.451093 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50404881-6346-43d1-b2fb-72129eaa7c6a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:45.451161 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50404881-6346-43d1-b2fb-72129eaa7c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:45.501365 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-50404881-6346-43d1-b2fb-72129eaa7c6a None None] MFA Rules not processed for user `8293dab3ffd647f5927d323572fdbdce`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:45.502593 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-50404881-6346-43d1-b2fb-72129eaa7c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:45.664169 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 839/1658] 10.222.0.50 () {58 vars in 1057 bytes} [Fri May 27 15:14:45 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:14:45.665201 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d139effc-0137-431b-b964-a6aac3e8f168 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:45.726016 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d139effc-0137-431b-b964-a6aac3e8f168 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:45.732188 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eb94d9-4a89-415c-ae92-94307ae7eae8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8293dab3ffd647f5927d323572fdbdce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:45.732865 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb94d9-4a89-415c-ae92-94307ae7eae8 service placement] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:45.733155 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb94d9-4a89-415c-ae92-94307ae7eae8 service placement] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:45.733437 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb94d9-4a89-415c-ae92-94307ae7eae8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:45.735760 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-59eb94d9-4a89-415c-ae92-94307ae7eae8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:45.760832 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eb94d9-4a89-415c-ae92-94307ae7eae8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:45.763550 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eb94d9-4a89-415c-ae92-94307ae7eae8 service placement] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:45.782975 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 820/1659] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:14:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:14:46.165577 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed4d52-83ca-4576-aa34-f11df70b89ca None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:46.165870 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed4d52-83ca-4576-aa34-f11df70b89ca None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:46.166596 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed4d52-83ca-4576-aa34-f11df70b89ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:46.202536 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-63ed4d52-83ca-4576-aa34-f11df70b89ca None None] MFA Rules not processed for user `0692caa0f5ce47099cf5e549e6be70ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:46.203981 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-63ed4d52-83ca-4576-aa34-f11df70b89ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:46.377209 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 840/1660] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:14:46 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:47.598183 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbc6d5-f97d-4275-9ff2-4061472ca8dd None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:47.599285 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbc6d5-f97d-4275-9ff2-4061472ca8dd None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:47.599823 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbc6d5-f97d-4275-9ff2-4061472ca8dd None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:47.604163 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 821/1661] 10.222.0.50 () {54 vars in 921 bytes} [Fri May 27 15:14:47 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:14:47.657457 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da0cb0-37bf-4f6e-8b76-b27fd83b2b85 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:47.658374 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da0cb0-37bf-4f6e-8b76-b27fd83b2b85 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:47.658807 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da0cb0-37bf-4f6e-8b76-b27fd83b2b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:47.712040 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-76da0cb0-37bf-4f6e-8b76-b27fd83b2b85 None None] MFA Rules not processed for user `8293dab3ffd647f5927d323572fdbdce`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:47.713296 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-76da0cb0-37bf-4f6e-8b76-b27fd83b2b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:47.739175 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 841/1662] 10.222.0.50 () {58 vars in 1018 bytes} [Fri May 27 15:14:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:47.791663 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fbc6d5-f97d-4275-9ff2-4061472ca8dd None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:47.813367 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fbc6d5-f97d-4275-9ff2-4061472ca8dd None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:47.832845 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e1f470-b3c2-4575-a489-77e6a0b279e5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8293dab3ffd647f5927d323572fdbdce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:47.833517 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1f470-b3c2-4575-a489-77e6a0b279e5 service placement] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:47.833832 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1f470-b3c2-4575-a489-77e6a0b279e5 service placement] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:47.844601 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e1f470-b3c2-4575-a489-77e6a0b279e5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:47.869000 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e1f470-b3c2-4575-a489-77e6a0b279e5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:47.875207 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e1f470-b3c2-4575-a489-77e6a0b279e5 service placement] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:47.919033 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 822/1663] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:14:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:14:48.717537 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76da0cb0-37bf-4f6e-8b76-b27fd83b2b85 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:48.839692 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:48.923541 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76da0cb0-37bf-4f6e-8b76-b27fd83b2b85 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:48.939392 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab03420-53cc-472d-83af-b61276654252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:48.940450 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab03420-53cc-472d-83af-b61276654252 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:48.940974 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab03420-53cc-472d-83af-b61276654252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:48.941894 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab03420-53cc-472d-83af-b61276654252 admin admin] PATH_INFO: `/v3/users/3d99e26b3cd04bba98f43712e0ae17e5` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:48.945038 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab03420-53cc-472d-83af-b61276654252 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d99e26b3cd04bba98f43712e0ae17e5)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:48.948430 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: WARNING py.warnings [None req-8ab03420-53cc-472d-83af-b61276654252 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 15:14:48.948430 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: warnings.warn(msg) May 27 15:14:48.948430 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: May 27 15:14:48.952894 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab03420-53cc-472d-83af-b61276654252 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:49.151952 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8ab03420-53cc-472d-83af-b61276654252 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d99e26b3cd04bba98f43712e0ae17e5'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:49.194160 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8ab03420-53cc-472d-83af-b61276654252 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d99e26b3cd04bba98f43712e0ae17e5'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:49.195240 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8ab03420-53cc-472d-83af-b61276654252 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d99e26b3cd04bba98f43712e0ae17e5'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:49.216756 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8ab03420-53cc-472d-83af-b61276654252 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d99e26b3cd04bba98f43712e0ae17e5'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:49.217527 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8ab03420-53cc-472d-83af-b61276654252 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d99e26b3cd04bba98f43712e0ae17e5'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:49.319144 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f1e1cc-fa32-4831-95f5-f1f3d547eb18 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:49.320208 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f1e1cc-fa32-4831-95f5-f1f3d547eb18 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:49.320956 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f1e1cc-fa32-4831-95f5-f1f3d547eb18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:49.361253 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a3f1e1cc-fa32-4831-95f5-f1f3d547eb18 None None] MFA Rules not processed for user `09c23d1c732c452a9597c647ad0241f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:49.362571 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a3f1e1cc-fa32-4831-95f5-f1f3d547eb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:49.367086 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 842/1664] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:14:48 2022] DELETE /identity/v3/users/3d99e26b3cd04bba98f43712e0ae17e5 => generated 0 bytes in 652 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:14:49.381177 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab03420-53cc-472d-83af-b61276654252 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:49.387983 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8ab03420-53cc-472d-83af-b61276654252 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:49.409857 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:49.512328 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 823/1665] 10.222.0.50 () {58 vars in 1009 bytes} [Fri May 27 15:14:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:14:49.608116 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab03420-53cc-472d-83af-b61276654252 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:49.630041 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a1fe7-4d9a-477f-98cf-df20658d824c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:49.631051 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a1fe7-4d9a-477f-98cf-df20658d824c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:49.631352 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a1fe7-4d9a-477f-98cf-df20658d824c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:49.631643 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a1fe7-4d9a-477f-98cf-df20658d824c admin admin] PATH_INFO: `/v3/users/5148faabb9414eeca471f00591eb167d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:49.633066 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a1fe7-4d9a-477f-98cf-df20658d824c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5148faabb9414eeca471f00591eb167d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:49.635349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a1fe7-4d9a-477f-98cf-df20658d824c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:49.767773 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-cd6a1fe7-4d9a-477f-98cf-df20658d824c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5148faabb9414eeca471f00591eb167d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:49.779297 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:49.788044 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-cd6a1fe7-4d9a-477f-98cf-df20658d824c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5148faabb9414eeca471f00591eb167d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:49.789376 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-cd6a1fe7-4d9a-477f-98cf-df20658d824c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5148faabb9414eeca471f00591eb167d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:49.801829 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f1e1cc-fa32-4831-95f5-f1f3d547eb18 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:49.803695 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a3f1e1cc-fa32-4831-95f5-f1f3d547eb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:49.829973 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-cd6a1fe7-4d9a-477f-98cf-df20658d824c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5148faabb9414eeca471f00591eb167d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:49.830968 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-cd6a1fe7-4d9a-477f-98cf-df20658d824c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5148faabb9414eeca471f00591eb167d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:49.868246 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 843/1666] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:14:49 2022] DELETE /identity/v3/users/5148faabb9414eeca471f00591eb167d => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:49.873321 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f4012-0378-4ea5-8f27-c9e8599be6b3 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:49.873807 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f4012-0378-4ea5-8f27-c9e8599be6b3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:49.874086 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7f4012-0378-4ea5-8f27-c9e8599be6b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:49.919814 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-cd7f4012-0378-4ea5-8f27-c9e8599be6b3 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:49.921962 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cd7f4012-0378-4ea5-8f27-c9e8599be6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:49.977072 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f1e1cc-fa32-4831-95f5-f1f3d547eb18 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:50.084759 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 844/1667] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:14:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:14:50.123072 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627c5405-3128-4e20-81fd-509b9e21e1f9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18903f2eab7c415096e075d47f0b9a4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:50.123929 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c5405-3128-4e20-81fd-509b9e21e1f9 service swift] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:50.124220 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c5405-3128-4e20-81fd-509b9e21e1f9 service swift] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:50.127546 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c5405-3128-4e20-81fd-509b9e21e1f9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:50.131686 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-627c5405-3128-4e20-81fd-509b9e21e1f9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:50.175352 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7f4012-0378-4ea5-8f27-c9e8599be6b3 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:50.181535 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cd7f4012-0378-4ea5-8f27-c9e8599be6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:50.309341 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627c5405-3128-4e20-81fd-509b9e21e1f9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:50.313315 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627c5405-3128-4e20-81fd-509b9e21e1f9 service swift] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:50.368567 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7f4012-0378-4ea5-8f27-c9e8599be6b3 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:50.387788 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 824/1668] 10.222.0.50 () {60 vars in 1384 bytes} [Fri May 27 15:14:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 27 15:14:50.407079 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01efbf-b576-41c3-b8e2-f8fd0f141353 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:50.407962 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01efbf-b576-41c3-b8e2-f8fd0f141353 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:50.408252 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01efbf-b576-41c3-b8e2-f8fd0f141353 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:50.410728 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01efbf-b576-41c3-b8e2-f8fd0f141353 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:50.462777 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a01efbf-b576-41c3-b8e2-f8fd0f141353 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:50.472122 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a01efbf-b576-41c3-b8e2-f8fd0f141353 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:50.520505 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 845/1669] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:14:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:14:50.888634 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627c5405-3128-4e20-81fd-509b9e21e1f9 service swift] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:50.890215 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-627c5405-3128-4e20-81fd-509b9e21e1f9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:50.930581 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627c5405-3128-4e20-81fd-509b9e21e1f9 service swift] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:50.956454 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ee106-ec2c-47a8-9a77-98bc97d4ee26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:50.957240 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ee106-ec2c-47a8-9a77-98bc97d4ee26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:50.957532 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ee106-ec2c-47a8-9a77-98bc97d4ee26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:50.957813 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ee106-ec2c-47a8-9a77-98bc97d4ee26 admin admin] PATH_INFO: `/v3/projects/cce3ede867a04744ae536cb737e14639` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:50.964046 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07ee106-ec2c-47a8-9a77-98bc97d4ee26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cce3ede867a04744ae536cb737e14639)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:50.966794 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: WARNING py.warnings [None req-b07ee106-ec2c-47a8-9a77-98bc97d4ee26 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 27 15:14:50.966794 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: warnings.warn(msg) May 27 15:14:50.966794 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: May 27 15:14:50.977491 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07ee106-ec2c-47a8-9a77-98bc97d4ee26 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:51.027077 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:51.274735 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b07ee106-ec2c-47a8-9a77-98bc97d4ee26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cce3ede867a04744ae536cb737e14639', 'request_id': 'req-b07ee106-ec2c-47a8-9a77-98bc97d4ee26'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:51.302896 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b07ee106-ec2c-47a8-9a77-98bc97d4ee26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cce3ede867a04744ae536cb737e14639', 'request_id': 'req-b07ee106-ec2c-47a8-9a77-98bc97d4ee26'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:51.384385 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b07ee106-ec2c-47a8-9a77-98bc97d4ee26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cce3ede867a04744ae536cb737e14639', 'request_id': 'req-b07ee106-ec2c-47a8-9a77-98bc97d4ee26'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:51.416190 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b07ee106-ec2c-47a8-9a77-98bc97d4ee26 admin admin] The token cache is being invalidate because project cce3ede867a04744ae536cb737e14639 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:14:51.416823 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b07ee106-ec2c-47a8-9a77-98bc97d4ee26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:51.420967 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b07ee106-ec2c-47a8-9a77-98bc97d4ee26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:51.423935 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 825/1670] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:14:50 2022] DELETE /identity/v3/projects/cce3ede867a04744ae536cb737e14639 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:51.428030 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01efbf-b576-41c3-b8e2-f8fd0f141353 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:51.446356 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01efbf-b576-41c3-b8e2-f8fd0f141353 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:51.681362 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f070f19-aa76-4911-82dd-b240b4fbbfdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:51.682450 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f070f19-aa76-4911-82dd-b240b4fbbfdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:51.682817 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f070f19-aa76-4911-82dd-b240b4fbbfdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:51.683056 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f070f19-aa76-4911-82dd-b240b4fbbfdc admin admin] PATH_INFO: `/v3/users/0692caa0f5ce47099cf5e549e6be70ac` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:51.686764 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f070f19-aa76-4911-82dd-b240b4fbbfdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=0692caa0f5ce47099cf5e549e6be70ac)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:51.701183 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f070f19-aa76-4911-82dd-b240b4fbbfdc admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:51.730173 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:51.854479 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8f070f19-aa76-4911-82dd-b240b4fbbfdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0692caa0f5ce47099cf5e549e6be70ac'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:51.895867 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8f070f19-aa76-4911-82dd-b240b4fbbfdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0692caa0f5ce47099cf5e549e6be70ac'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:51.900024 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8f070f19-aa76-4911-82dd-b240b4fbbfdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0692caa0f5ce47099cf5e549e6be70ac'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:51.944439 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8f070f19-aa76-4911-82dd-b240b4fbbfdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0692caa0f5ce47099cf5e549e6be70ac'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:51.944439 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8f070f19-aa76-4911-82dd-b240b4fbbfdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0692caa0f5ce47099cf5e549e6be70ac'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:52.035367 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ee106-ec2c-47a8-9a77-98bc97d4ee26 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:52.037526 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b07ee106-ec2c-47a8-9a77-98bc97d4ee26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:52.054574 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 846/1671] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:14:51 2022] DELETE /identity/v3/users/0692caa0f5ce47099cf5e549e6be70ac => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:52.068225 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ee106-ec2c-47a8-9a77-98bc97d4ee26 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:52.233451 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bdf7c7-7184-4b87-b9c1-e4c68274b6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:52.234834 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bdf7c7-7184-4b87-b9c1-e4c68274b6a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:52.235538 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bdf7c7-7184-4b87-b9c1-e4c68274b6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:52.236192 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bdf7c7-7184-4b87-b9c1-e4c68274b6a9 admin admin] PATH_INFO: `/v3/users/37f29ccc2165414e91f51a38d2d918dd` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:52.238605 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bdf7c7-7184-4b87-b9c1-e4c68274b6a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37f29ccc2165414e91f51a38d2d918dd)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:52.241528 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bdf7c7-7184-4b87-b9c1-e4c68274b6a9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:52.347982 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e7bdf7c7-7184-4b87-b9c1-e4c68274b6a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37f29ccc2165414e91f51a38d2d918dd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:52.368681 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e7bdf7c7-7184-4b87-b9c1-e4c68274b6a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37f29ccc2165414e91f51a38d2d918dd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:52.401034 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e7bdf7c7-7184-4b87-b9c1-e4c68274b6a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37f29ccc2165414e91f51a38d2d918dd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:52.403166 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e7bdf7c7-7184-4b87-b9c1-e4c68274b6a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37f29ccc2165414e91f51a38d2d918dd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:52.404261 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e7bdf7c7-7184-4b87-b9c1-e4c68274b6a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37f29ccc2165414e91f51a38d2d918dd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:52.466449 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 826/1672] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:14:52 2022] DELETE /identity/v3/users/37f29ccc2165414e91f51a38d2d918dd => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:14:52.470662 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381cb197-7753-40aa-afb5-735812306d8c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:52.473260 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381cb197-7753-40aa-afb5-735812306d8c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:52.473799 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381cb197-7753-40aa-afb5-735812306d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:52.537665 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-381cb197-7753-40aa-afb5-735812306d8c None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:52.539567 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-381cb197-7753-40aa-afb5-735812306d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:52.718270 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 847/1673] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:14:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:14:52.748178 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bdf7c7-7184-4b87-b9c1-e4c68274b6a9 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:52.749977 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e7bdf7c7-7184-4b87-b9c1-e4c68274b6a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:52.787809 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:53.003258 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bdf7c7-7184-4b87-b9c1-e4c68274b6a9 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:53.019328 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381cb197-7753-40aa-afb5-735812306d8c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:53.019803 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfb2fb0-2d84-4f7d-96b7-171f4ff769c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:53.020512 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb2fb0-2d84-4f7d-96b7-171f4ff769c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:53.021014 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-381cb197-7753-40aa-afb5-735812306d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:53.024709 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb2fb0-2d84-4f7d-96b7-171f4ff769c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:53.025474 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb2fb0-2d84-4f7d-96b7-171f4ff769c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:53.046049 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfb2fb0-2d84-4f7d-96b7-171f4ff769c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:53.048681 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfb2fb0-2d84-4f7d-96b7-171f4ff769c3 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:53.075569 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 827/1674] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:14:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:14:53.155888 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381cb197-7753-40aa-afb5-735812306d8c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:53.160982 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed872c02-60ed-4bad-994d-cf3af6fb21f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:53.161652 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed872c02-60ed-4bad-994d-cf3af6fb21f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:53.162557 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed872c02-60ed-4bad-994d-cf3af6fb21f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:53.162557 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed872c02-60ed-4bad-994d-cf3af6fb21f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:53.167592 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ed872c02-60ed-4bad-994d-cf3af6fb21f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:53.340822 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:53.398003 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed872c02-60ed-4bad-994d-cf3af6fb21f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:53.400191 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed872c02-60ed-4bad-994d-cf3af6fb21f7 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:53.419578 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfb2fb0-2d84-4f7d-96b7-171f4ff769c3 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:53.421396 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2dfb2fb0-2d84-4f7d-96b7-171f4ff769c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:53.453338 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 848/1675] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:14:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:14:53.456018 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfb2fb0-2d84-4f7d-96b7-171f4ff769c3 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:53.487923 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd1b64-a2c5-494a-9b32-a15144e698b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:53.489613 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd1b64-a2c5-494a-9b32-a15144e698b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:53.490277 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd1b64-a2c5-494a-9b32-a15144e698b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:53.492657 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd1b64-a2c5-494a-9b32-a15144e698b4 admin admin] PATH_INFO: `/v3/projects/efb8b11b65d04ce5a16afcc8ab579931` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:53.496723 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fd1b64-a2c5-494a-9b32-a15144e698b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efb8b11b65d04ce5a16afcc8ab579931)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:53.506756 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fd1b64-a2c5-494a-9b32-a15144e698b4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:53.650962 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f7fd1b64-a2c5-494a-9b32-a15144e698b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efb8b11b65d04ce5a16afcc8ab579931', 'request_id': 'req-f7fd1b64-a2c5-494a-9b32-a15144e698b4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:53.667684 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f7fd1b64-a2c5-494a-9b32-a15144e698b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efb8b11b65d04ce5a16afcc8ab579931', 'request_id': 'req-f7fd1b64-a2c5-494a-9b32-a15144e698b4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:53.684674 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f7fd1b64-a2c5-494a-9b32-a15144e698b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efb8b11b65d04ce5a16afcc8ab579931', 'request_id': 'req-f7fd1b64-a2c5-494a-9b32-a15144e698b4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:53.712959 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f7fd1b64-a2c5-494a-9b32-a15144e698b4 admin admin] The token cache is being invalidate because project efb8b11b65d04ce5a16afcc8ab579931 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:14:53.713225 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f7fd1b64-a2c5-494a-9b32-a15144e698b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:53.713919 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f7fd1b64-a2c5-494a-9b32-a15144e698b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:53.723560 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 828/1676] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:14:53 2022] DELETE /identity/v3/projects/efb8b11b65d04ce5a16afcc8ab579931 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:53.728620 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da7f03a-7cbe-4be9-8359-72322b78f02a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:53.729134 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da7f03a-7cbe-4be9-8359-72322b78f02a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:53.729229 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da7f03a-7cbe-4be9-8359-72322b78f02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:53.776344 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1da7f03a-7cbe-4be9-8359-72322b78f02a None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:53.784419 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1da7f03a-7cbe-4be9-8359-72322b78f02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:53.840072 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e7e1ce-7e19-4ca5-bc73-82b01c2a6af1 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:53.840380 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e7e1ce-7e19-4ca5-bc73-82b01c2a6af1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:53.840668 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e7e1ce-7e19-4ca5-bc73-82b01c2a6af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:53.910786 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-39e7e1ce-7e19-4ca5-bc73-82b01c2a6af1 None None] MFA Rules not processed for user `f874edc08438434189f1d14d79291487`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:53.915653 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-39e7e1ce-7e19-4ca5-bc73-82b01c2a6af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:54.015895 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 849/1677] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:14:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:54.024508 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e248a69-0407-4b88-ba62-8c23aec76156 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:54.024808 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e248a69-0407-4b88-ba62-8c23aec76156 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:54.025096 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e248a69-0407-4b88-ba62-8c23aec76156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:54.070681 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5e248a69-0407-4b88-ba62-8c23aec76156 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:54.072125 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5e248a69-0407-4b88-ba62-8c23aec76156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:54.125541 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 829/1678] 10.222.0.27 () {58 vars in 1015 bytes} [Fri May 27 15:14:53 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:54.125939 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 850/1679] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:14:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:54.145363 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e248a69-0407-4b88-ba62-8c23aec76156 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:54.147458 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e7e1ce-7e19-4ca5-bc73-82b01c2a6af1 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:54.149133 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5e248a69-0407-4b88-ba62-8c23aec76156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:54.192685 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e7e1ce-7e19-4ca5-bc73-82b01c2a6af1 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:54.208066 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e248a69-0407-4b88-ba62-8c23aec76156 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:54.213415 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b81d91-e1ca-447b-bdf4-1c6f9b64618f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:54.214435 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b81d91-e1ca-447b-bdf4-1c6f9b64618f admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:54.218587 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b81d91-e1ca-447b-bdf4-1c6f9b64618f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:54.219043 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b81d91-e1ca-447b-bdf4-1c6f9b64618f admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:54.222890 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e4a69b-4fe0-4e39-95ed-81546aedf245 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:54.223134 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b81d91-e1ca-447b-bdf4-1c6f9b64618f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:14:54.223500 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b81d91-e1ca-447b-bdf4-1c6f9b64618f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:54.224196 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4a69b-4fe0-4e39-95ed-81546aedf245 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:54.224623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4a69b-4fe0-4e39-95ed-81546aedf245 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:54.225100 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4a69b-4fe0-4e39-95ed-81546aedf245 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:54.228666 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b81d91-e1ca-447b-bdf4-1c6f9b64618f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:54.243072 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e4a69b-4fe0-4e39-95ed-81546aedf245 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:54.244899 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e4a69b-4fe0-4e39-95ed-81546aedf245 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:54.260456 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 851/1680] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:14:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:14:54.264824 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e4a69b-4fe0-4e39-95ed-81546aedf245 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:54.268854 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e3e4a69b-4fe0-4e39-95ed-81546aedf245 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:54.295202 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 830/1681] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:14:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:14:54.299486 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b81d91-e1ca-447b-bdf4-1c6f9b64618f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:54.309817 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e4a69b-4fe0-4e39-95ed-81546aedf245 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:54.322946 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bd187e-1dbb-4a30-ae59-e8c46623e8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:54.325248 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd187e-1dbb-4a30-ae59-e8c46623e8e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:54.326375 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd187e-1dbb-4a30-ae59-e8c46623e8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:54.327490 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bd187e-1dbb-4a30-ae59-e8c46623e8e6 admin admin] PATH_INFO: `/v3/projects/435f9c8ceeb248ac94af711caa2249dc` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:54.331818 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bd187e-1dbb-4a30-ae59-e8c46623e8e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=435f9c8ceeb248ac94af711caa2249dc)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:54.339200 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bd187e-1dbb-4a30-ae59-e8c46623e8e6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:54.361565 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b81d91-e1ca-447b-bdf4-1c6f9b64618f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:54.392984 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49de62e8-92e4-4b49-a1e6-0077990444b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:54.396138 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de62e8-92e4-4b49-a1e6-0077990444b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:54.396431 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de62e8-92e4-4b49-a1e6-0077990444b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:54.396717 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de62e8-92e4-4b49-a1e6-0077990444b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:54.401460 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49de62e8-92e4-4b49-a1e6-0077990444b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:54.404596 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49de62e8-92e4-4b49-a1e6-0077990444b8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:54.488967 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-81bd187e-1dbb-4a30-ae59-e8c46623e8e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '435f9c8ceeb248ac94af711caa2249dc', 'request_id': 'req-81bd187e-1dbb-4a30-ae59-e8c46623e8e6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:54.509473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-81bd187e-1dbb-4a30-ae59-e8c46623e8e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '435f9c8ceeb248ac94af711caa2249dc', 'request_id': 'req-81bd187e-1dbb-4a30-ae59-e8c46623e8e6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:54.518960 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 831/1682] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:14:54 2022] POST /identity/v3/projects => generated 371 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:14:54.519445 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-81bd187e-1dbb-4a30-ae59-e8c46623e8e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '435f9c8ceeb248ac94af711caa2249dc', 'request_id': 'req-81bd187e-1dbb-4a30-ae59-e8c46623e8e6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:54.522291 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49de62e8-92e4-4b49-a1e6-0077990444b8 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:54.560833 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-81bd187e-1dbb-4a30-ae59-e8c46623e8e6 admin admin] The token cache is being invalidate because project 435f9c8ceeb248ac94af711caa2249dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:14:54.561152 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-81bd187e-1dbb-4a30-ae59-e8c46623e8e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:54.563359 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-81bd187e-1dbb-4a30-ae59-e8c46623e8e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:54.576800 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 852/1683] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:14:54 2022] DELETE /identity/v3/projects/435f9c8ceeb248ac94af711caa2249dc => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:54.578364 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361aa18b-7df9-484d-b635-7f9669173682 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:54.578657 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361aa18b-7df9-484d-b635-7f9669173682 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:54.578964 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361aa18b-7df9-484d-b635-7f9669173682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:54.582863 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:54.648927 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-361aa18b-7df9-484d-b635-7f9669173682 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:54.656213 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-361aa18b-7df9-484d-b635-7f9669173682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:54.665535 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49de62e8-92e4-4b49-a1e6-0077990444b8 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:54.671658 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51797883-0428-454b-8b55-fc6070bfcddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:54.673098 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51797883-0428-454b-8b55-fc6070bfcddd admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:54.674086 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51797883-0428-454b-8b55-fc6070bfcddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:54.674685 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51797883-0428-454b-8b55-fc6070bfcddd admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:54.675718 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51797883-0428-454b-8b55-fc6070bfcddd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:54.678253 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51797883-0428-454b-8b55-fc6070bfcddd admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:54.692421 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 853/1684] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:14:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:54.706899 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6b259-4b3e-4239-8aec-5d5bc0dc287d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:54.707098 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6b259-4b3e-4239-8aec-5d5bc0dc287d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:54.720040 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6b259-4b3e-4239-8aec-5d5bc0dc287d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:54.735875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 832/1685] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:14:54 2022] POST /identity/v3/users => generated 286 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:14:54.740549 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51797883-0428-454b-8b55-fc6070bfcddd admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:54.746487 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-51797883-0428-454b-8b55-fc6070bfcddd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:54.778948 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51797883-0428-454b-8b55-fc6070bfcddd admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:54.798088 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f8d6b259-4b3e-4239-8aec-5d5bc0dc287d None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:54.801580 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f8d6b259-4b3e-4239-8aec-5d5bc0dc287d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:54.811346 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8255e1a-5334-421e-a6c0-02713a8b675c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:54.817024 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8255e1a-5334-421e-a6c0-02713a8b675c admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:54.817915 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8255e1a-5334-421e-a6c0-02713a8b675c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:54.819666 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8255e1a-5334-421e-a6c0-02713a8b675c admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:54.821774 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8255e1a-5334-421e-a6c0-02713a8b675c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:54.828926 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8255e1a-5334-421e-a6c0-02713a8b675c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:54.844419 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 854/1686] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:14:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:54.848417 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d6b259-4b3e-4239-8aec-5d5bc0dc287d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:54.873899 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d6b259-4b3e-4239-8aec-5d5bc0dc287d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:54.888745 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 833/1687] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:14:54 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:14:54.893643 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8255e1a-5334-421e-a6c0-02713a8b675c admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:54.896781 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e70f6-aa65-450a-a6da-65d4c46a3228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:54.897477 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e70f6-aa65-450a-a6da-65d4c46a3228 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:54.897771 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e70f6-aa65-450a-a6da-65d4c46a3228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:54.898057 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e70f6-aa65-450a-a6da-65d4c46a3228 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:54.898802 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e70f6-aa65-450a-a6da-65d4c46a3228 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:14:54.899140 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e70f6-aa65-450a-a6da-65d4c46a3228 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:54.900774 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e70f6-aa65-450a-a6da-65d4c46a3228 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:54.923564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8255e1a-5334-421e-a6c0-02713a8b675c admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:54.943722 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc82c47f-db3f-4aa3-8361-b2ec221b60db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:54.945908 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82c47f-db3f-4aa3-8361-b2ec221b60db admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:54.946451 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82c47f-db3f-4aa3-8361-b2ec221b60db admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:54.947860 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc82c47f-db3f-4aa3-8361-b2ec221b60db admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:54.949323 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc82c47f-db3f-4aa3-8361-b2ec221b60db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:54.951894 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 855/1688] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:14:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:14:54.952961 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc82c47f-db3f-4aa3-8361-b2ec221b60db admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:54.959207 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e70f6-aa65-450a-a6da-65d4c46a3228 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:54.994840 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e70f6-aa65-450a-a6da-65d4c46a3228 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:54.995083 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 834/1689] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:14:54 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:14:54.996156 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc82c47f-db3f-4aa3-8361-b2ec221b60db admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:55.011766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c364f2-dd0a-4dca-93ac-3bfa7a149171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:55.012473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c364f2-dd0a-4dca-93ac-3bfa7a149171 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:55.012767 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c364f2-dd0a-4dca-93ac-3bfa7a149171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:55.013047 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c364f2-dd0a-4dca-93ac-3bfa7a149171 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:55.013762 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c364f2-dd0a-4dca-93ac-3bfa7a149171 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:55.020143 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c364f2-dd0a-4dca-93ac-3bfa7a149171 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:55.033899 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc82c47f-db3f-4aa3-8361-b2ec221b60db admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:55.064676 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde57cc7-efbd-40fd-9283-fa75c238621f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:55.065878 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde57cc7-efbd-40fd-9283-fa75c238621f admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:55.066500 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde57cc7-efbd-40fd-9283-fa75c238621f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:55.067023 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde57cc7-efbd-40fd-9283-fa75c238621f admin admin] PATH_INFO: `/v3/projects/703969aecb654b3f9fff32e41d8f8f09/users/a9a2b077f6704778bf4090b555a02c60/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:55.106631 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde57cc7-efbd-40fd-9283-fa75c238621f admin admin] RBAC: Authorizing `identity:create_grant(project_id=703969aecb654b3f9fff32e41d8f8f09, user_id=a9a2b077f6704778bf4090b555a02c60, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:55.124672 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 856/1690] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:14:54 2022] POST /identity/v3/projects => generated 379 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:14:55.125622 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c364f2-dd0a-4dca-93ac-3bfa7a149171 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:55.140453 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:55.162602 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde57cc7-efbd-40fd-9283-fa75c238621f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:55.199934 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 835/1691] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:14:54 2022] PUT /identity/v3/projects/703969aecb654b3f9fff32e41d8f8f09/users/a9a2b077f6704778bf4090b555a02c60/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:55.201760 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5912f65-b3e6-4cef-b493-66b82bdcaa49 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:55.202167 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5912f65-b3e6-4cef-b493-66b82bdcaa49 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:55.202547 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5912f65-b3e6-4cef-b493-66b82bdcaa49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:55.313103 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b5912f65-b3e6-4cef-b493-66b82bdcaa49 None None] MFA Rules not processed for user `a9a2b077f6704778bf4090b555a02c60`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:55.314490 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b5912f65-b3e6-4cef-b493-66b82bdcaa49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:55.403031 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c364f2-dd0a-4dca-93ac-3bfa7a149171 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:55.544968 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 836/1692] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:14:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:55.546051 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5912f65-b3e6-4cef-b493-66b82bdcaa49 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:55.578277 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55015266-dc23-4e62-9d51-4b009c1bc678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:55.579495 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55015266-dc23-4e62-9d51-4b009c1bc678 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:55.580051 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55015266-dc23-4e62-9d51-4b009c1bc678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:55.580494 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55015266-dc23-4e62-9d51-4b009c1bc678 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:55.581419 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55015266-dc23-4e62-9d51-4b009c1bc678 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:55.584436 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55015266-dc23-4e62-9d51-4b009c1bc678 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:55.644742 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 857/1693] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:14:55 2022] POST /identity/v3/users => generated 290 bytes in 519 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:14:55.645664 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55015266-dc23-4e62-9d51-4b009c1bc678 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:55.681156 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:55.682267 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55015266-dc23-4e62-9d51-4b009c1bc678 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:55.699827 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a0f293-db99-45dd-bdd4-98899be236a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:55.700676 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a0f293-db99-45dd-bdd4-98899be236a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:55.700968 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a0f293-db99-45dd-bdd4-98899be236a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:55.701245 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a0f293-db99-45dd-bdd4-98899be236a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:55.701981 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a0f293-db99-45dd-bdd4-98899be236a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:55.703646 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a0f293-db99-45dd-bdd4-98899be236a5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:55.728367 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5912f65-b3e6-4cef-b493-66b82bdcaa49 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:55.745652 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 858/1694] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:14:55 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:14:55.751825 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a0f293-db99-45dd-bdd4-98899be236a5 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:55.753659 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8a8638-1edb-4859-a616-3b7dfaf29186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:55.754613 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a8638-1edb-4859-a616-3b7dfaf29186 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:55.755007 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a8638-1edb-4859-a616-3b7dfaf29186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:55.755157 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8a8638-1edb-4859-a616-3b7dfaf29186 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:55.756057 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8a8638-1edb-4859-a616-3b7dfaf29186 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:55.765415 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a0f293-db99-45dd-bdd4-98899be236a5 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:55.776780 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8a8638-1edb-4859-a616-3b7dfaf29186 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:55.785939 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f7b1a-baaf-43d5-b20d-5818f62bcb0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:55.786699 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f7b1a-baaf-43d5-b20d-5818f62bcb0d admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:55.787011 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f7b1a-baaf-43d5-b20d-5818f62bcb0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:55.787345 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f7b1a-baaf-43d5-b20d-5818f62bcb0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:55.788143 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4f7b1a-baaf-43d5-b20d-5818f62bcb0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:55.789952 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4f7b1a-baaf-43d5-b20d-5818f62bcb0d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:55.830051 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 859/1695] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:14:55 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:14:55.834365 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f7b1a-baaf-43d5-b20d-5818f62bcb0d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:55.851308 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4f7b1a-baaf-43d5-b20d-5818f62bcb0d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:55.858420 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe6459-d83b-49a9-bb1c-6dd058d2250c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:55.859620 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe6459-d83b-49a9-bb1c-6dd058d2250c admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:55.860068 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe6459-d83b-49a9-bb1c-6dd058d2250c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:55.860461 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe6459-d83b-49a9-bb1c-6dd058d2250c admin admin] PATH_INFO: `/v3/projects/763b86e473ee403091dca92587e2f7e9/users/9f027a39f90a469bb21dd585fc02ab4c/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:55.923288 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 837/1696] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:14:55 2022] POST /identity/v3/projects => generated 373 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:14:55.924393 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8a8638-1edb-4859-a616-3b7dfaf29186 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:55.927712 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fe6459-d83b-49a9-bb1c-6dd058d2250c admin admin] RBAC: Authorizing `identity:create_grant(project_id=763b86e473ee403091dca92587e2f7e9, user_id=9f027a39f90a469bb21dd585fc02ab4c, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:55.939728 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fe6459-d83b-49a9-bb1c-6dd058d2250c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:55.991275 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 860/1697] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:14:55 2022] PUT /identity/v3/projects/763b86e473ee403091dca92587e2f7e9/users/9f027a39f90a469bb21dd585fc02ab4c/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:56.004698 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd76c52-1557-410d-af48-7afce10e1246 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:56.005300 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd76c52-1557-410d-af48-7afce10e1246 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:56.005748 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd76c52-1557-410d-af48-7afce10e1246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:56.088141 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7bd76c52-1557-410d-af48-7afce10e1246 None None] MFA Rules not processed for user `9f027a39f90a469bb21dd585fc02ab4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:56.089522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7bd76c52-1557-410d-af48-7afce10e1246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:56.175393 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8a8638-1edb-4859-a616-3b7dfaf29186 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:56.256214 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:56.317255 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 861/1698] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:14:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:56.321473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd76c52-1557-410d-af48-7afce10e1246 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:56.327667 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7bd76c52-1557-410d-af48-7afce10e1246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:56.345497 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266878fb-7ae1-43f5-bc3f-5088586cc297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:56.346264 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266878fb-7ae1-43f5-bc3f-5088586cc297 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:56.346563 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266878fb-7ae1-43f5-bc3f-5088586cc297 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:56.346879 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266878fb-7ae1-43f5-bc3f-5088586cc297 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:56.353733 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266878fb-7ae1-43f5-bc3f-5088586cc297 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:56.356819 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266878fb-7ae1-43f5-bc3f-5088586cc297 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:56.371512 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd76c52-1557-410d-af48-7afce10e1246 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:56.394055 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a9ba6e-a28d-473b-a7f5-caaccf76cdda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:56.394995 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9ba6e-a28d-473b-a7f5-caaccf76cdda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:56.395284 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9ba6e-a28d-473b-a7f5-caaccf76cdda admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:56.395567 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9ba6e-a28d-473b-a7f5-caaccf76cdda admin admin] PATH_INFO: `/v3/users/f0f600b3b58b43a0829c7430bb3eaf4d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:56.406556 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a9ba6e-a28d-473b-a7f5-caaccf76cdda admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0f600b3b58b43a0829c7430bb3eaf4d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:56.409672 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a9ba6e-a28d-473b-a7f5-caaccf76cdda admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:56.435211 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 838/1699] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:14:55 2022] POST /identity/v3/users => generated 293 bytes in 511 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:14:56.436076 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266878fb-7ae1-43f5-bc3f-5088586cc297 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:56.481145 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266878fb-7ae1-43f5-bc3f-5088586cc297 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:56.501523 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9553fb-293d-440b-ba43-129ae7566106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:56.504275 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9553fb-293d-440b-ba43-129ae7566106 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:56.505223 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9553fb-293d-440b-ba43-129ae7566106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:56.505612 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9553fb-293d-440b-ba43-129ae7566106 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:56.507405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9553fb-293d-440b-ba43-129ae7566106 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:56.514358 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9553fb-293d-440b-ba43-129ae7566106 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:56.603033 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-33a9ba6e-a28d-473b-a7f5-caaccf76cdda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0f600b3b58b43a0829c7430bb3eaf4d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:56.635263 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 839/1700] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:14:56 2022] POST /identity/v3/projects => generated 379 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:14:56.637595 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9553fb-293d-440b-ba43-129ae7566106 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:56.640974 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-33a9ba6e-a28d-473b-a7f5-caaccf76cdda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0f600b3b58b43a0829c7430bb3eaf4d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:56.642585 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-33a9ba6e-a28d-473b-a7f5-caaccf76cdda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0f600b3b58b43a0829c7430bb3eaf4d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:56.665162 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-33a9ba6e-a28d-473b-a7f5-caaccf76cdda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0f600b3b58b43a0829c7430bb3eaf4d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:56.667732 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-33a9ba6e-a28d-473b-a7f5-caaccf76cdda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0f600b3b58b43a0829c7430bb3eaf4d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:56.719605 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 862/1701] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:14:56 2022] DELETE /identity/v3/users/f0f600b3b58b43a0829c7430bb3eaf4d => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:56.725924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0454bcd1-6d40-4269-9f14-cab881790ff5 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:56.726221 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0454bcd1-6d40-4269-9f14-cab881790ff5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:56.727556 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0454bcd1-6d40-4269-9f14-cab881790ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:56.807702 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:56.816184 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0454bcd1-6d40-4269-9f14-cab881790ff5 None None] MFA Rules not processed for user `387048f42b634693a7e827a8d88ef89b`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:56.820112 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0454bcd1-6d40-4269-9f14-cab881790ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:56.870334 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9553fb-293d-440b-ba43-129ae7566106 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:57.052282 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 863/1702] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:14:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:57.053232 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0454bcd1-6d40-4269-9f14-cab881790ff5 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:57.061410 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0454bcd1-6d40-4269-9f14-cab881790ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:57.156905 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:57.222026 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d14af82-e81e-4ab9-be77-615bda91ec9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:57.222730 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d14af82-e81e-4ab9-be77-615bda91ec9a admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:57.223045 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d14af82-e81e-4ab9-be77-615bda91ec9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:57.223367 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d14af82-e81e-4ab9-be77-615bda91ec9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:57.224125 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d14af82-e81e-4ab9-be77-615bda91ec9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:57.225787 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d14af82-e81e-4ab9-be77-615bda91ec9a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:57.231841 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0454bcd1-6d40-4269-9f14-cab881790ff5 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:57.239057 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3861e6a-b42e-494d-8032-f0c9a30d1040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:57.240194 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3861e6a-b42e-494d-8032-f0c9a30d1040 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:57.240732 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3861e6a-b42e-494d-8032-f0c9a30d1040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:57.242662 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3861e6a-b42e-494d-8032-f0c9a30d1040 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:57.243424 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3861e6a-b42e-494d-8032-f0c9a30d1040 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:57.245461 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3861e6a-b42e-494d-8032-f0c9a30d1040 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:57.247054 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 840/1703] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:14:56 2022] GET /identity/v3/roles => generated 1420 bytes in 611 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:14:57.247992 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d14af82-e81e-4ab9-be77-615bda91ec9a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:57.264126 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3d14af82-e81e-4ab9-be77-615bda91ec9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:57.281311 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d14af82-e81e-4ab9-be77-615bda91ec9a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:57.305586 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 864/1704] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:14:57 2022] POST /identity/v3/users => generated 296 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:14:57.305586 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc5e4fd-9f38-4502-aa90-72e19ef03066 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:57.305586 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc5e4fd-9f38-4502-aa90-72e19ef03066 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:57.305586 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc5e4fd-9f38-4502-aa90-72e19ef03066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:57.307739 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c254b-55fb-4539-b8ab-9d857cf059bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:57.309452 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c254b-55fb-4539-b8ab-9d857cf059bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:57.309739 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c254b-55fb-4539-b8ab-9d857cf059bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:57.310243 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989c254b-55fb-4539-b8ab-9d857cf059bd admin admin] PATH_INFO: `/v3/projects/d3a001740e9e476c9cee17b42c1c7816` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:57.311992 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989c254b-55fb-4539-b8ab-9d857cf059bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3a001740e9e476c9cee17b42c1c7816)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:57.313954 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989c254b-55fb-4539-b8ab-9d857cf059bd admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:57.368003 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9dc5e4fd-9f38-4502-aa90-72e19ef03066 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:57.372684 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9dc5e4fd-9f38-4502-aa90-72e19ef03066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:57.399239 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 865/1705] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:14:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:57.404751 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d766a-2ac4-416d-bbbf-9e76f55494d1 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:57.405111 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d766a-2ac4-416d-bbbf-9e76f55494d1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:57.405817 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338d766a-2ac4-416d-bbbf-9e76f55494d1 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:57.411864 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 866/1706] 10.222.0.50 () {54 vars in 921 bytes} [Fri May 27 15:14:57 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:14:57.414932 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d766a-2ac4-416d-bbbf-9e76f55494d1 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:57.420242 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-338d766a-2ac4-416d-bbbf-9e76f55494d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:57.432011 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-989c254b-55fb-4539-b8ab-9d857cf059bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3a001740e9e476c9cee17b42c1c7816', 'request_id': 'req-989c254b-55fb-4539-b8ab-9d857cf059bd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:57.449690 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-989c254b-55fb-4539-b8ab-9d857cf059bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3a001740e9e476c9cee17b42c1c7816', 'request_id': 'req-989c254b-55fb-4539-b8ab-9d857cf059bd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:57.465760 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-989c254b-55fb-4539-b8ab-9d857cf059bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3a001740e9e476c9cee17b42c1c7816', 'request_id': 'req-989c254b-55fb-4539-b8ab-9d857cf059bd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:57.520608 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-989c254b-55fb-4539-b8ab-9d857cf059bd admin admin] The token cache is being invalidate because project d3a001740e9e476c9cee17b42c1c7816 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:14:57.521832 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-989c254b-55fb-4539-b8ab-9d857cf059bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:57.524925 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-989c254b-55fb-4539-b8ab-9d857cf059bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:57.533236 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 841/1707] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:14:57 2022] DELETE /identity/v3/projects/d3a001740e9e476c9cee17b42c1c7816 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:57.534500 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c254b-55fb-4539-b8ab-9d857cf059bd admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:57.540057 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-989c254b-55fb-4539-b8ab-9d857cf059bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:57.586042 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338d766a-2ac4-416d-bbbf-9e76f55494d1 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:57.610482 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a008df-407a-4c64-bac4-4daba2ba85f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '451a9232f95e494683d81b6078a318d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:57.612284 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a008df-407a-4c64-bac4-4daba2ba85f4 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:57.612578 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a008df-407a-4c64-bac4-4daba2ba85f4 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:57.613984 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a008df-407a-4c64-bac4-4daba2ba85f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:57.618923 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a5a008df-407a-4c64-bac4-4daba2ba85f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:57.690625 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989c254b-55fb-4539-b8ab-9d857cf059bd admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:57.697455 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db282ebd-fa4b-4564-b43d-ecb7264fcdbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:57.698503 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db282ebd-fa4b-4564-b43d-ecb7264fcdbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:57.699911 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db282ebd-fa4b-4564-b43d-ecb7264fcdbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:57.700336 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db282ebd-fa4b-4564-b43d-ecb7264fcdbe admin admin] PATH_INFO: `/v3/projects/e43c1071d395445db48c7a5c4adb24b0/users/39abb37fd92a4985a1c694d9e83ae8c8/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:57.724402 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db282ebd-fa4b-4564-b43d-ecb7264fcdbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=e43c1071d395445db48c7a5c4adb24b0, user_id=39abb37fd92a4985a1c694d9e83ae8c8, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:57.727807 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db282ebd-fa4b-4564-b43d-ecb7264fcdbe admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:57.744049 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:57.754415 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 842/1708] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:14:57 2022] PUT /identity/v3/projects/e43c1071d395445db48c7a5c4adb24b0/users/39abb37fd92a4985a1c694d9e83ae8c8/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:57.760172 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db282ebd-fa4b-4564-b43d-ecb7264fcdbe admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:57.803797 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-db282ebd-fa4b-4564-b43d-ecb7264fcdbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:57.939940 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a008df-407a-4c64-bac4-4daba2ba85f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:57.940139 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db282ebd-fa4b-4564-b43d-ecb7264fcdbe admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:57.949001 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a008df-407a-4c64-bac4-4daba2ba85f4 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:57.958218 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ecc7fa-3d88-4589-b8a7-42b2fb86316d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:57.960917 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ecc7fa-3d88-4589-b8a7-42b2fb86316d admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:57.963628 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ecc7fa-3d88-4589-b8a7-42b2fb86316d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:57.964203 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ecc7fa-3d88-4589-b8a7-42b2fb86316d admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:57.987699 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ecc7fa-3d88-4589-b8a7-42b2fb86316d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:57.992434 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ecc7fa-3d88-4589-b8a7-42b2fb86316d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:58.000078 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 867/1709] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:14:57 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:14:58.004434 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a008df-407a-4c64-bac4-4daba2ba85f4 service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:58.012159 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a5a008df-407a-4c64-bac4-4daba2ba85f4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:58.085269 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 843/1710] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:14:57 2022] GET /identity/v3/roles => generated 1420 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:14:58.090305 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de6db90-05ee-48ec-a2c9-706bf58e736e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:58.093957 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de6db90-05ee-48ec-a2c9-706bf58e736e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:58.094326 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de6db90-05ee-48ec-a2c9-706bf58e736e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:58.166317 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-6de6db90-05ee-48ec-a2c9-706bf58e736e None None] MFA Rules not processed for user `451a9232f95e494683d81b6078a318d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:58.166508 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a008df-407a-4c64-bac4-4daba2ba85f4 service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:58.171566 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6de6db90-05ee-48ec-a2c9-706bf58e736e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:58.207314 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:58.287042 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e234d2f-5073-44bd-b9de-4bc6561b4a9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:58.287780 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e234d2f-5073-44bd-b9de-4bc6561b4a9b service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:58.288192 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e234d2f-5073-44bd-b9de-4bc6561b4a9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:58.288414 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e234d2f-5073-44bd-b9de-4bc6561b4a9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:58.294577 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3e234d2f-5073-44bd-b9de-4bc6561b4a9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:58.324750 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e234d2f-5073-44bd-b9de-4bc6561b4a9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:58.335729 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e234d2f-5073-44bd-b9de-4bc6561b4a9b service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:58.354927 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 868/1711] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:14:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:14:58.358877 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b8c90-dd8c-4fad-a8d7-05f8f271f555 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:58.359282 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b8c90-dd8c-4fad-a8d7-05f8f271f555 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:58.359733 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695b8c90-dd8c-4fad-a8d7-05f8f271f555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:58.370567 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 844/1712] 10.222.0.50 () {58 vars in 1018 bytes} [Fri May 27 15:14:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:58.371809 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de6db90-05ee-48ec-a2c9-706bf58e736e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:58.405856 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-695b8c90-dd8c-4fad-a8d7-05f8f271f555 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:58.409170 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de6db90-05ee-48ec-a2c9-706bf58e736e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:58.412380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-695b8c90-dd8c-4fad-a8d7-05f8f271f555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:58.432118 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98439b74-0e02-4904-84e2-709dc775f417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:58.436804 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98439b74-0e02-4904-84e2-709dc775f417 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:58.439827 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98439b74-0e02-4904-84e2-709dc775f417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:58.440342 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98439b74-0e02-4904-84e2-709dc775f417 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:58.442314 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98439b74-0e02-4904-84e2-709dc775f417 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:58.443566 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 869/1713] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:14:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:58.446525 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e9923-5ab4-4846-812d-2530d9ab2edc None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:58.446817 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e9923-5ab4-4846-812d-2530d9ab2edc None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:58.449873 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98439b74-0e02-4904-84e2-709dc775f417 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:58.450063 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e9923-5ab4-4846-812d-2530d9ab2edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:58.525693 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 845/1714] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:14:58 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:14:58.533996 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98439b74-0e02-4904-84e2-709dc775f417 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:58.563933 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-fa0e9923-5ab4-4846-812d-2530d9ab2edc None None] MFA Rules not processed for user `5a990918de6d409a949616da90de6a5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:58.565175 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fa0e9923-5ab4-4846-812d-2530d9ab2edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:58.574440 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98439b74-0e02-4904-84e2-709dc775f417 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:58.596348 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e85847-b1ee-4dca-bdcd-df62c9738441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:58.600385 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e85847-b1ee-4dca-bdcd-df62c9738441 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:58.601856 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e85847-b1ee-4dca-bdcd-df62c9738441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:58.602467 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e85847-b1ee-4dca-bdcd-df62c9738441 admin admin] PATH_INFO: `/v3/projects/83f83d3ce2934cbdbb5d680757391e81/users/196c2b91af4e4c588238489784255907/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:58.658154 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e85847-b1ee-4dca-bdcd-df62c9738441 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83f83d3ce2934cbdbb5d680757391e81, user_id=196c2b91af4e4c588238489784255907, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:58.662486 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e85847-b1ee-4dca-bdcd-df62c9738441 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:58.691919 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 846/1715] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:14:58 2022] PUT /identity/v3/projects/83f83d3ce2934cbdbb5d680757391e81/users/196c2b91af4e4c588238489784255907/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:58.693722 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e85847-b1ee-4dca-bdcd-df62c9738441 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:58.697247 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-25e85847-b1ee-4dca-bdcd-df62c9738441 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:58.794439 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 870/1716] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:14:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:58.795957 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10415dba-3ad1-4103-9cf7-2cc04fa2b8c3 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:58.796249 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10415dba-3ad1-4103-9cf7-2cc04fa2b8c3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:58.796802 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10415dba-3ad1-4103-9cf7-2cc04fa2b8c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:58.834155 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:58.848005 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-10415dba-3ad1-4103-9cf7-2cc04fa2b8c3 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:14:58.849338 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-10415dba-3ad1-4103-9cf7-2cc04fa2b8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:58.915676 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e85847-b1ee-4dca-bdcd-df62c9738441 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:58.933811 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58142288-fb11-4f37-9af9-bfd1e6e85b9e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '451a9232f95e494683d81b6078a318d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:58.938172 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58142288-fb11-4f37-9af9-bfd1e6e85b9e service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:58.939234 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58142288-fb11-4f37-9af9-bfd1e6e85b9e service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:58.950117 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58142288-fb11-4f37-9af9-bfd1e6e85b9e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:58.963009 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58142288-fb11-4f37-9af9-bfd1e6e85b9e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:58.967176 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58142288-fb11-4f37-9af9-bfd1e6e85b9e service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:58.995549 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 847/1717] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:14:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:14:58.998757 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58142288-fb11-4f37-9af9-bfd1e6e85b9e service nova] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:59.016839 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 871/1718] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:14:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:14:59.020683 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10415dba-3ad1-4103-9cf7-2cc04fa2b8c3 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:59.026975 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-10415dba-3ad1-4103-9cf7-2cc04fa2b8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:59.028944 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58142288-fb11-4f37-9af9-bfd1e6e85b9e service nova] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:59.042752 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c34bca-1c12-44e2-bdde-643908f74f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:59.043312 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c34bca-1c12-44e2-bdde-643908f74f7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:59.044526 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c34bca-1c12-44e2-bdde-643908f74f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:59.044917 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c34bca-1c12-44e2-bdde-643908f74f7b admin admin] PATH_INFO: `/v3/projects/e43c1071d395445db48c7a5c4adb24b0/users/39abb37fd92a4985a1c694d9e83ae8c8/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:59.051905 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c34bca-1c12-44e2-bdde-643908f74f7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e43c1071d395445db48c7a5c4adb24b0, user_id=39abb37fd92a4985a1c694d9e83ae8c8, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:59.060172 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c34bca-1c12-44e2-bdde-643908f74f7b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:59.072654 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10415dba-3ad1-4103-9cf7-2cc04fa2b8c3 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:59.094880 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 848/1719] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:14:58 2022] PUT /identity/v3/projects/e43c1071d395445db48c7a5c4adb24b0/users/39abb37fd92a4985a1c694d9e83ae8c8/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:59.094880 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c34bca-1c12-44e2-bdde-643908f74f7b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:59.259277 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:59.296145 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a3e7ee-ef26-4a79-bd51-523c9839c5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:59.299547 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3e7ee-ef26-4a79-bd51-523c9839c5f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:59.300199 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3e7ee-ef26-4a79-bd51-523c9839c5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:59.300621 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3e7ee-ef26-4a79-bd51-523c9839c5f0 admin admin] PATH_INFO: `/v3/users/c899cd80a3dc4f15be32d3f26dce4b1f` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:59.304750 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a3e7ee-ef26-4a79-bd51-523c9839c5f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c899cd80a3dc4f15be32d3f26dce4b1f)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:59.311723 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c34bca-1c12-44e2-bdde-643908f74f7b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:59.316692 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a3e7ee-ef26-4a79-bd51-523c9839c5f0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:59.335053 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160f8a1f-c5e8-4c05-a57e-8b33fe2937d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:59.335939 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160f8a1f-c5e8-4c05-a57e-8b33fe2937d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:59.342177 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160f8a1f-c5e8-4c05-a57e-8b33fe2937d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:59.345478 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160f8a1f-c5e8-4c05-a57e-8b33fe2937d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:59.346403 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160f8a1f-c5e8-4c05-a57e-8b33fe2937d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:59.355255 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160f8a1f-c5e8-4c05-a57e-8b33fe2937d2 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:59.398400 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 849/1720] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:14:59 2022] GET /identity/v3/roles => generated 1420 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:14:59.399605 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160f8a1f-c5e8-4c05-a57e-8b33fe2937d2 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:59.406277 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-160f8a1f-c5e8-4c05-a57e-8b33fe2937d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:59.434648 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-86a3e7ee-ef26-4a79-bd51-523c9839c5f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c899cd80a3dc4f15be32d3f26dce4b1f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:59.467325 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-86a3e7ee-ef26-4a79-bd51-523c9839c5f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c899cd80a3dc4f15be32d3f26dce4b1f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:59.471011 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-86a3e7ee-ef26-4a79-bd51-523c9839c5f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c899cd80a3dc4f15be32d3f26dce4b1f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:59.496314 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-86a3e7ee-ef26-4a79-bd51-523c9839c5f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c899cd80a3dc4f15be32d3f26dce4b1f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:59.497666 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-86a3e7ee-ef26-4a79-bd51-523c9839c5f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c899cd80a3dc4f15be32d3f26dce4b1f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:14:59.584567 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 872/1721] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:14:59 2022] DELETE /identity/v3/users/c899cd80a3dc4f15be32d3f26dce4b1f => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:14:59.585514 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a3e7ee-ef26-4a79-bd51-523c9839c5f0 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:14:59.598017 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-86a3e7ee-ef26-4a79-bd51-523c9839c5f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:59.640440 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160f8a1f-c5e8-4c05-a57e-8b33fe2937d2 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:59.831640 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:14:59.852152 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a3e7ee-ef26-4a79-bd51-523c9839c5f0 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:14:59.867832 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2420abb-743b-47c2-8f47-1923bd42fb38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:59.869144 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2420abb-743b-47c2-8f47-1923bd42fb38 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:59.870591 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2420abb-743b-47c2-8f47-1923bd42fb38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:59.871071 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2420abb-743b-47c2-8f47-1923bd42fb38 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:59.872130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2420abb-743b-47c2-8f47-1923bd42fb38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:14:59.872551 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2420abb-743b-47c2-8f47-1923bd42fb38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:14:59.883318 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc090217-506f-4f6a-b7ea-b56808f8627d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '451a9232f95e494683d81b6078a318d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:14:59.887817 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc090217-506f-4f6a-b7ea-b56808f8627d service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:59.888392 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc090217-506f-4f6a-b7ea-b56808f8627d service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:59.888668 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2420abb-743b-47c2-8f47-1923bd42fb38 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:14:59.889172 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc090217-506f-4f6a-b7ea-b56808f8627d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:14:59.895122 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fc090217-506f-4f6a-b7ea-b56808f8627d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:14:59.950839 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 873/1722] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:14:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:14:59.953796 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a98882-06b0-4ede-acb4-b58b57511826 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:14:59.954093 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a98882-06b0-4ede-acb4-b58b57511826 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:14:59.954381 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a98882-06b0-4ede-acb4-b58b57511826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:00.044340 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc090217-506f-4f6a-b7ea-b56808f8627d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:00.052437 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc090217-506f-4f6a-b7ea-b56808f8627d service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:00.075229 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e8a98882-06b0-4ede-acb4-b58b57511826 None None] MFA Rules not processed for user `39abb37fd92a4985a1c694d9e83ae8c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:00.078303 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e8a98882-06b0-4ede-acb4-b58b57511826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:00.080729 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 850/1723] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:14:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 682 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:00.100292 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc090217-506f-4f6a-b7ea-b56808f8627d service nova] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:00.108666 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fc090217-506f-4f6a-b7ea-b56808f8627d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:00.145433 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc090217-506f-4f6a-b7ea-b56808f8627d service nova] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:00.161080 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45228312-fd83-4711-b0c3-24651b150d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:00.164586 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45228312-fd83-4711-b0c3-24651b150d75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:00.165742 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45228312-fd83-4711-b0c3-24651b150d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:00.167207 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45228312-fd83-4711-b0c3-24651b150d75 admin admin] PATH_INFO: `/v3/projects/83f83d3ce2934cbdbb5d680757391e81/users/196c2b91af4e4c588238489784255907/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:00.174269 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45228312-fd83-4711-b0c3-24651b150d75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83f83d3ce2934cbdbb5d680757391e81, user_id=196c2b91af4e4c588238489784255907, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:00.181419 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45228312-fd83-4711-b0c3-24651b150d75 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:00.215270 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 851/1724] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:00 2022] PUT /identity/v3/projects/83f83d3ce2934cbdbb5d680757391e81/users/196c2b91af4e4c588238489784255907/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:00.217139 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45228312-fd83-4711-b0c3-24651b150d75 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:00.222342 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-45228312-fd83-4711-b0c3-24651b150d75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:00.309320 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:00.397960 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 874/1725] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:14:59 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 447 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:00.399534 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b25db-6ca0-4757-a752-4cd24a4758ab None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:00.399827 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b25db-6ca0-4757-a752-4cd24a4758ab None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:00.403254 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b25db-6ca0-4757-a752-4cd24a4758ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:00.472710 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e78b25db-6ca0-4757-a752-4cd24a4758ab None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:00.473963 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e78b25db-6ca0-4757-a752-4cd24a4758ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:00.487993 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45228312-fd83-4711-b0c3-24651b150d75 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:00.512085 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b270e0de-06ee-4049-a691-84a3a884f389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:00.513005 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270e0de-06ee-4049-a691-84a3a884f389 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:00.513294 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270e0de-06ee-4049-a691-84a3a884f389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:00.513574 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270e0de-06ee-4049-a691-84a3a884f389 admin admin] PATH_INFO: `/v3/projects/c5744215ba8d4f439159f9532f7d88b3` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:00.514724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 875/1726] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:00.516587 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b25db-6ca0-4757-a752-4cd24a4758ab None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:00.521988 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b270e0de-06ee-4049-a691-84a3a884f389 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5744215ba8d4f439159f9532f7d88b3)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:00.526683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b270e0de-06ee-4049-a691-84a3a884f389 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:00.556054 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b25db-6ca0-4757-a752-4cd24a4758ab None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:00.571723 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8f52c3-35c7-4c6b-8d00-8ba40531836b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:00.572751 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f52c3-35c7-4c6b-8d00-8ba40531836b admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:00.573084 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f52c3-35c7-4c6b-8d00-8ba40531836b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:00.573370 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f52c3-35c7-4c6b-8d00-8ba40531836b admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:00.574599 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8f52c3-35c7-4c6b-8d00-8ba40531836b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:00.580720 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8f52c3-35c7-4c6b-8d00-8ba40531836b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:00.669432 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 876/1727] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:15:00 2022] POST /identity/v3/projects => generated 379 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:00.679590 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a042d-a328-4fb8-906a-4735f42471ae None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:00.679725 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a042d-a328-4fb8-906a-4735f42471ae None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:00.680001 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53a042d-a328-4fb8-906a-4735f42471ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:00.705949 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b270e0de-06ee-4049-a691-84a3a884f389 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5744215ba8d4f439159f9532f7d88b3', 'request_id': 'req-b270e0de-06ee-4049-a691-84a3a884f389'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:00.728655 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b270e0de-06ee-4049-a691-84a3a884f389 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5744215ba8d4f439159f9532f7d88b3', 'request_id': 'req-b270e0de-06ee-4049-a691-84a3a884f389'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:00.754212 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b270e0de-06ee-4049-a691-84a3a884f389 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5744215ba8d4f439159f9532f7d88b3', 'request_id': 'req-b270e0de-06ee-4049-a691-84a3a884f389'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:00.772005 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c53a042d-a328-4fb8-906a-4735f42471ae None None] MFA Rules not processed for user `196c2b91af4e4c588238489784255907`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:00.773262 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c53a042d-a328-4fb8-906a-4735f42471ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:00.789170 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b270e0de-06ee-4049-a691-84a3a884f389 admin admin] The token cache is being invalidate because project c5744215ba8d4f439159f9532f7d88b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:15:00.789620 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b270e0de-06ee-4049-a691-84a3a884f389 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:00.791636 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b270e0de-06ee-4049-a691-84a3a884f389 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:00.800700 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 852/1728] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:15:00 2022] DELETE /identity/v3/projects/c5744215ba8d4f439159f9532f7d88b3 => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:00.801679 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b270e0de-06ee-4049-a691-84a3a884f389 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:00.804704 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b270e0de-06ee-4049-a691-84a3a884f389 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:00.846423 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:00.999438 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b270e0de-06ee-4049-a691-84a3a884f389 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:01.015284 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 877/1729] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:00 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:01.016754 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a042d-a328-4fb8-906a-4735f42471ae None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:01.028639 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c53a042d-a328-4fb8-906a-4735f42471ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:01.122245 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e7052-b6d4-4dab-b369-f5d18d6d28ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:01.123204 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e7052-b6d4-4dab-b369-f5d18d6d28ca service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:01.123631 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e7052-b6d4-4dab-b369-f5d18d6d28ca service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:01.124137 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9e7052-b6d4-4dab-b369-f5d18d6d28ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:01.132498 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cc9e7052-b6d4-4dab-b369-f5d18d6d28ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:01.248889 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53a042d-a328-4fb8-906a-4735f42471ae None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:01.265895 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73abdf6a-b574-4810-a41a-293993b6b96e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:01.266580 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73abdf6a-b574-4810-a41a-293993b6b96e service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:01.266883 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73abdf6a-b574-4810-a41a-293993b6b96e service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:01.267171 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73abdf6a-b574-4810-a41a-293993b6b96e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:01.273419 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-73abdf6a-b574-4810-a41a-293993b6b96e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:01.345657 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9e7052-b6d4-4dab-b369-f5d18d6d28ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:01.351737 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9e7052-b6d4-4dab-b369-f5d18d6d28ca service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:01.385155 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 853/1730] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:00 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 582 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:01.386103 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e7052-b6d4-4dab-b369-f5d18d6d28ca service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:01.391096 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cc9e7052-b6d4-4dab-b369-f5d18d6d28ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:01.450657 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:01.495674 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73abdf6a-b574-4810-a41a-293993b6b96e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:01.502537 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73abdf6a-b574-4810-a41a-293993b6b96e service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:01.529784 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 878/1731] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:01.531359 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c13ce-491b-474b-b4c3-1e52ce0bca8b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:01.531654 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c13ce-491b-474b-b4c3-1e52ce0bca8b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:01.531967 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c13ce-491b-474b-b4c3-1e52ce0bca8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:01.599847 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6a9c13ce-491b-474b-b4c3-1e52ce0bca8b None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:01.599847 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6a9c13ce-491b-474b-b4c3-1e52ce0bca8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:01.601785 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9e7052-b6d4-4dab-b369-f5d18d6d28ca service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:01.619253 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8c0077-4df6-416d-a121-f7e21ba1ba3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:01.619954 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c0077-4df6-416d-a121-f7e21ba1ba3e admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:01.620317 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c0077-4df6-416d-a121-f7e21ba1ba3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:01.620736 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8c0077-4df6-416d-a121-f7e21ba1ba3e admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:01.622783 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8c0077-4df6-416d-a121-f7e21ba1ba3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:01.631624 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8c0077-4df6-416d-a121-f7e21ba1ba3e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:01.641628 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 879/1732] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:01.641628 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9c13ce-491b-474b-b4c3-1e52ce0bca8b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:01.643231 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6a9c13ce-491b-474b-b4c3-1e52ce0bca8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:01.688561 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9c13ce-491b-474b-b4c3-1e52ce0bca8b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:01.692169 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 854/1733] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:15:01 2022] POST /identity/v3/users => generated 290 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:01.699069 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfadab8-9f56-4991-a7e9-8a3b00ad1e97 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:01.699982 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed6e323-2ecf-44b6-ab6b-1c44949982dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:01.700727 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6e323-2ecf-44b6-ab6b-1c44949982dd service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:01.701049 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfadab8-9f56-4991-a7e9-8a3b00ad1e97 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:01.701470 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6e323-2ecf-44b6-ab6b-1c44949982dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:01.701848 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfadab8-9f56-4991-a7e9-8a3b00ad1e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:01.702208 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6e323-2ecf-44b6-ab6b-1c44949982dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:01.732876 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed6e323-2ecf-44b6-ab6b-1c44949982dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:01.735356 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed6e323-2ecf-44b6-ab6b-1c44949982dd service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:01.739360 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:01.765841 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-bdfadab8-9f56-4991-a7e9-8a3b00ad1e97 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:01.771443 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bdfadab8-9f56-4991-a7e9-8a3b00ad1e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:01.776701 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 880/1734] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:01 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:01.779889 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed6e323-2ecf-44b6-ab6b-1c44949982dd service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:01.796548 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 855/1735] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:01.801193 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfadab8-9f56-4991-a7e9-8a3b00ad1e97 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:01.809820 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed6e323-2ecf-44b6-ab6b-1c44949982dd service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:01.833936 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5651e1-4555-4b1e-bf8d-a00e69a8fa3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:01.835518 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfadab8-9f56-4991-a7e9-8a3b00ad1e97 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:01.838673 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5651e1-4555-4b1e-bf8d-a00e69a8fa3e admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:01.840718 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5651e1-4555-4b1e-bf8d-a00e69a8fa3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:01.841219 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5651e1-4555-4b1e-bf8d-a00e69a8fa3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:01.842214 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5651e1-4555-4b1e-bf8d-a00e69a8fa3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:01.847764 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5651e1-4555-4b1e-bf8d-a00e69a8fa3e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:01.858261 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd1f4b1-775e-4606-bf15-fe46bf49f81c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:01.858883 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd1f4b1-775e-4606-bf15-fe46bf49f81c service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:01.858977 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd1f4b1-775e-4606-bf15-fe46bf49f81c service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:01.859293 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd1f4b1-775e-4606-bf15-fe46bf49f81c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:01.866585 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cdd1f4b1-775e-4606-bf15-fe46bf49f81c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:01.894664 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 881/1736] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:01.899957 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5651e1-4555-4b1e-bf8d-a00e69a8fa3e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:01.933092 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5651e1-4555-4b1e-bf8d-a00e69a8fa3e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:01.947543 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e83d82a-2397-43d0-8361-d42bf78d13fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:01.951102 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83d82a-2397-43d0-8361-d42bf78d13fa admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:01.951757 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83d82a-2397-43d0-8361-d42bf78d13fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:01.952075 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e83d82a-2397-43d0-8361-d42bf78d13fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:01.953479 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e83d82a-2397-43d0-8361-d42bf78d13fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:15:01.954022 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e83d82a-2397-43d0-8361-d42bf78d13fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:01.957710 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e83d82a-2397-43d0-8361-d42bf78d13fa admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:01.983081 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 882/1737] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:15:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:15:01.984646 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e83d82a-2397-43d0-8361-d42bf78d13fa admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:02.008808 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e83d82a-2397-43d0-8361-d42bf78d13fa admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:02.015162 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c98f85-985a-4817-9baa-512c08540b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:02.016711 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c98f85-985a-4817-9baa-512c08540b5c admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:02.017003 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c98f85-985a-4817-9baa-512c08540b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:02.017316 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c98f85-985a-4817-9baa-512c08540b5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:02.018609 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c98f85-985a-4817-9baa-512c08540b5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:02.020561 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c98f85-985a-4817-9baa-512c08540b5c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:02.033165 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd1f4b1-775e-4606-bf15-fe46bf49f81c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:02.034781 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd1f4b1-775e-4606-bf15-fe46bf49f81c service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:02.037484 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 883/1738] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:02.038543 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c98f85-985a-4817-9baa-512c08540b5c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:02.051815 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 856/1739] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:02.052680 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd1f4b1-775e-4606-bf15-fe46bf49f81c service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:02.071209 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c98f85-985a-4817-9baa-512c08540b5c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:02.087773 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd1f4b1-775e-4606-bf15-fe46bf49f81c service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:02.089436 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c98c55-4e51-4692-a645-88ce16ab4bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:02.090834 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c98c55-4e51-4692-a645-88ce16ab4bef admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:02.093345 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c98c55-4e51-4692-a645-88ce16ab4bef admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:02.093783 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b8216-3fc5-42f9-b6b0-e29f985667f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:02.094823 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b8216-3fc5-42f9-b6b0-e29f985667f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:02.095450 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b8216-3fc5-42f9-b6b0-e29f985667f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:02.095852 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b8216-3fc5-42f9-b6b0-e29f985667f0 admin admin] PATH_INFO: `/v3/projects/59c6c742cd434430a8e7b7e1fa954335/users/0b42cad9fde34170b9ad058e24e761e5/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:02.096705 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c98c55-4e51-4692-a645-88ce16ab4bef admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:02.097952 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c98c55-4e51-4692-a645-88ce16ab4bef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:02.103175 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c98c55-4e51-4692-a645-88ce16ab4bef admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:02.122664 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019b8216-3fc5-42f9-b6b0-e29f985667f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59c6c742cd434430a8e7b7e1fa954335, user_id=0b42cad9fde34170b9ad058e24e761e5, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:02.134043 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019b8216-3fc5-42f9-b6b0-e29f985667f0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:02.168588 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 884/1740] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:15:02 2022] POST /identity/v3/projects => generated 373 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:02.173106 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c98c55-4e51-4692-a645-88ce16ab4bef admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:02.200025 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 857/1741] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:02 2022] PUT /identity/v3/projects/59c6c742cd434430a8e7b7e1fa954335/users/0b42cad9fde34170b9ad058e24e761e5/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:02.205020 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1679fe-9d31-4ab7-a551-aea895d8a18b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:02.206750 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1679fe-9d31-4ab7-a551-aea895d8a18b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:02.206928 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1679fe-9d31-4ab7-a551-aea895d8a18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:02.280417 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8e1679fe-9d31-4ab7-a551-aea895d8a18b None None] MFA Rules not processed for user `0b42cad9fde34170b9ad058e24e761e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:02.286168 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8e1679fe-9d31-4ab7-a551-aea895d8a18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:02.477652 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 858/1742] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:02.478825 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1679fe-9d31-4ab7-a551-aea895d8a18b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:02.506152 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1679fe-9d31-4ab7-a551-aea895d8a18b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:02.523351 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596e276f-07c3-45a7-b0ce-3cb13827a2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:02.524523 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e276f-07c3-45a7-b0ce-3cb13827a2f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:02.524998 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e276f-07c3-45a7-b0ce-3cb13827a2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:02.525502 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e276f-07c3-45a7-b0ce-3cb13827a2f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:02.526374 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596e276f-07c3-45a7-b0ce-3cb13827a2f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:02.529514 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596e276f-07c3-45a7-b0ce-3cb13827a2f6 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:02.556849 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c98c55-4e51-4692-a645-88ce16ab4bef admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:02.582128 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731390fb-92aa-468a-b542-e6676fdaf186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:02.582899 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731390fb-92aa-468a-b542-e6676fdaf186 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:02.583167 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731390fb-92aa-468a-b542-e6676fdaf186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:02.583433 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731390fb-92aa-468a-b542-e6676fdaf186 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:02.584186 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731390fb-92aa-468a-b542-e6676fdaf186 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:02.591809 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731390fb-92aa-468a-b542-e6676fdaf186 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:02.627368 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 859/1743] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:15:02 2022] POST /identity/v3/projects => generated 379 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:02.628386 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596e276f-07c3-45a7-b0ce-3cb13827a2f6 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:02.669755 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 885/1744] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:15:02 2022] POST /identity/v3/users => generated 287 bytes in 497 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:02.676222 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731390fb-92aa-468a-b542-e6676fdaf186 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:02.706550 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731390fb-92aa-468a-b542-e6676fdaf186 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:02.812777 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:02.878448 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596e276f-07c3-45a7-b0ce-3cb13827a2f6 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:02.901819 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a5b3fa-5c3a-43b4-a715-e51232d9575d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:02.903528 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5b3fa-5c3a-43b4-a715-e51232d9575d admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:02.903956 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5b3fa-5c3a-43b4-a715-e51232d9575d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:02.904130 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5b3fa-5c3a-43b4-a715-e51232d9575d admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:02.904845 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a5b3fa-5c3a-43b4-a715-e51232d9575d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:02.907100 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a5b3fa-5c3a-43b4-a715-e51232d9575d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:02.959743 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb09b76-494a-4754-8a47-a69037691887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:02.960461 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb09b76-494a-4754-8a47-a69037691887 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:02.960745 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb09b76-494a-4754-8a47-a69037691887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:02.961025 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb09b76-494a-4754-8a47-a69037691887 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:02.961784 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb09b76-494a-4754-8a47-a69037691887 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:02.963701 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb09b76-494a-4754-8a47-a69037691887 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:02.985529 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 860/1745] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:15:02 2022] POST /identity/v3/users => generated 296 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:02.986629 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a5b3fa-5c3a-43b4-a715-e51232d9575d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:02.992376 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:03.009975 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 886/1746] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:03.009975 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb09b76-494a-4754-8a47-a69037691887 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:03.046804 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb09b76-494a-4754-8a47-a69037691887 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:03.064656 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e916e89-39eb-4cb5-a519-e9e5601ff205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:03.075660 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e916e89-39eb-4cb5-a519-e9e5601ff205 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:03.075952 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e916e89-39eb-4cb5-a519-e9e5601ff205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:03.076265 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e916e89-39eb-4cb5-a519-e9e5601ff205 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:03.077095 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e916e89-39eb-4cb5-a519-e9e5601ff205 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:03.078987 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e916e89-39eb-4cb5-a519-e9e5601ff205 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:03.130931 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 887/1747] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:03 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:03.132029 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e916e89-39eb-4cb5-a519-e9e5601ff205 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:03.158661 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e916e89-39eb-4cb5-a519-e9e5601ff205 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:03.176678 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b8e8a1-2118-4a74-804f-98d92c914fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:03.178137 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b8e8a1-2118-4a74-804f-98d92c914fc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:03.178429 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b8e8a1-2118-4a74-804f-98d92c914fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:03.179185 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b8e8a1-2118-4a74-804f-98d92c914fc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:03.179949 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b8e8a1-2118-4a74-804f-98d92c914fc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:03.188017 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b8e8a1-2118-4a74-804f-98d92c914fc8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:03.216234 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a5b3fa-5c3a-43b4-a715-e51232d9575d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:03.240487 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 888/1748] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:03 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:03.241462 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b8e8a1-2118-4a74-804f-98d92c914fc8 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:03.249536 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b97ad1-d9ae-449d-8e59-27318ffd203c tempest-VolumeQuotaClassesTest-876872140 tempest-VolumeQuotaClassesTest-876872140-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '196c2b91af4e4c588238489784255907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83f83d3ce2934cbdbb5d680757391e81', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:03.250227 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b97ad1-d9ae-449d-8e59-27318ffd203c tempest-VolumeQuotaClassesTest-876872140 tempest-VolumeQuotaClassesTest-876872140-project-admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:03.250511 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b97ad1-d9ae-449d-8e59-27318ffd203c tempest-VolumeQuotaClassesTest-876872140 tempest-VolumeQuotaClassesTest-876872140-project-admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:03.250790 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b97ad1-d9ae-449d-8e59-27318ffd203c tempest-VolumeQuotaClassesTest-876872140 tempest-VolumeQuotaClassesTest-876872140-project-admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:03.251560 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b97ad1-d9ae-449d-8e59-27318ffd203c tempest-VolumeQuotaClassesTest-876872140 tempest-VolumeQuotaClassesTest-876872140-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:15:03.251874 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b97ad1-d9ae-449d-8e59-27318ffd203c tempest-VolumeQuotaClassesTest-876872140 tempest-VolumeQuotaClassesTest-876872140-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:03.253493 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b97ad1-d9ae-449d-8e59-27318ffd203c tempest-VolumeQuotaClassesTest-876872140 tempest-VolumeQuotaClassesTest-876872140-project-admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:03.269362 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b8e8a1-2118-4a74-804f-98d92c914fc8 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:03.284109 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 861/1749] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:15:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:15:03.285136 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b97ad1-d9ae-449d-8e59-27318ffd203c tempest-VolumeQuotaClassesTest-876872140 tempest-VolumeQuotaClassesTest-876872140-project-admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:03.287937 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9d0c4e-40e8-477b-a394-a50b34e4233b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:03.289060 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9d0c4e-40e8-477b-a394-a50b34e4233b admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:03.289356 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9d0c4e-40e8-477b-a394-a50b34e4233b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:03.289630 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9d0c4e-40e8-477b-a394-a50b34e4233b admin admin] PATH_INFO: `/v3/projects/d7439103a9514268ab6be38eb1860b7f/users/b179bcde825a465ab113c2fe2bc62661/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:03.310467 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b97ad1-d9ae-449d-8e59-27318ffd203c tempest-VolumeQuotaClassesTest-876872140 tempest-VolumeQuotaClassesTest-876872140-project-admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:03.321332 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb74acfb-fc2b-4879-8f1d-21ee01152cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:03.322072 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74acfb-fc2b-4879-8f1d-21ee01152cbd admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:03.322349 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74acfb-fc2b-4879-8f1d-21ee01152cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:03.322958 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74acfb-fc2b-4879-8f1d-21ee01152cbd admin admin] PATH_INFO: `/v3/projects/8293dd408a05464f987aa79eed0d09fa/users/61dd84b3080d445abb173ab3642f6886/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:03.334151 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9d0c4e-40e8-477b-a394-a50b34e4233b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7439103a9514268ab6be38eb1860b7f, user_id=b179bcde825a465ab113c2fe2bc62661, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:03.352836 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9d0c4e-40e8-477b-a394-a50b34e4233b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:03.374818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb74acfb-fc2b-4879-8f1d-21ee01152cbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8293dd408a05464f987aa79eed0d09fa, user_id=61dd84b3080d445abb173ab3642f6886, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:03.378664 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb74acfb-fc2b-4879-8f1d-21ee01152cbd admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:03.410761 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 862/1750] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:03 2022] PUT /identity/v3/projects/8293dd408a05464f987aa79eed0d09fa/users/61dd84b3080d445abb173ab3642f6886/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:03.411872 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb74acfb-fc2b-4879-8f1d-21ee01152cbd admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:03.441948 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 889/1751] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:03 2022] PUT /identity/v3/projects/d7439103a9514268ab6be38eb1860b7f/users/b179bcde825a465ab113c2fe2bc62661/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:03.443538 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0172251-3537-4311-b609-ba6be539796e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:03.443855 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0172251-3537-4311-b609-ba6be539796e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:03.449848 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0172251-3537-4311-b609-ba6be539796e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:03.556674 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b0172251-3537-4311-b609-ba6be539796e None None] MFA Rules not processed for user `61dd84b3080d445abb173ab3642f6886`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:03.560734 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b0172251-3537-4311-b609-ba6be539796e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:03.563176 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb74acfb-fc2b-4879-8f1d-21ee01152cbd admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:03.568558 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4290ea7d-d4b8-4a3d-8a34-2e4e49f2ce31 tempest-VolumeQuotaClassesTest-876872140 tempest-VolumeQuotaClassesTest-876872140-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '196c2b91af4e4c588238489784255907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83f83d3ce2934cbdbb5d680757391e81', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:03.569587 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290ea7d-d4b8-4a3d-8a34-2e4e49f2ce31 tempest-VolumeQuotaClassesTest-876872140 tempest-VolumeQuotaClassesTest-876872140-project-admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:03.570018 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290ea7d-d4b8-4a3d-8a34-2e4e49f2ce31 tempest-VolumeQuotaClassesTest-876872140 tempest-VolumeQuotaClassesTest-876872140-project-admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:03.570431 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290ea7d-d4b8-4a3d-8a34-2e4e49f2ce31 tempest-VolumeQuotaClassesTest-876872140 tempest-VolumeQuotaClassesTest-876872140-project-admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:03.571364 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4290ea7d-d4b8-4a3d-8a34-2e4e49f2ce31 tempest-VolumeQuotaClassesTest-876872140 tempest-VolumeQuotaClassesTest-876872140-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:03.573545 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4290ea7d-d4b8-4a3d-8a34-2e4e49f2ce31 tempest-VolumeQuotaClassesTest-876872140 tempest-VolumeQuotaClassesTest-876872140-project-admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:03.651842 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 863/1752] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:15:03 2022] POST /identity/v3/projects => generated 354 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:03.653690 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4290ea7d-d4b8-4a3d-8a34-2e4e49f2ce31 tempest-VolumeQuotaClassesTest-876872140 tempest-VolumeQuotaClassesTest-876872140-project-admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:03.719348 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:03.767100 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 890/1753] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:03.769418 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0172251-3537-4311-b609-ba6be539796e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:03.809360 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0172251-3537-4311-b609-ba6be539796e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:03.814322 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4290ea7d-d4b8-4a3d-8a34-2e4e49f2ce31 tempest-VolumeQuotaClassesTest-876872140 tempest-VolumeQuotaClassesTest-876872140-project-admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:03.831103 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ed2c7-8257-476f-89ce-6890010ee4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:03.831799 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ed2c7-8257-476f-89ce-6890010ee4ef admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:03.832123 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ed2c7-8257-476f-89ce-6890010ee4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:03.832409 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ed2c7-8257-476f-89ce-6890010ee4ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:03.836178 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22ed2c7-8257-476f-89ce-6890010ee4ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:03.837854 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22ed2c7-8257-476f-89ce-6890010ee4ef admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:03.877248 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 864/1754] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:03 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:03.878330 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ed2c7-8257-476f-89ce-6890010ee4ef admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:03.890221 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c22ed2c7-8257-476f-89ce-6890010ee4ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:03.986700 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c849f7-7c77-4779-8b09-215471ec2b90 tempest-VolumeQuotaClassesTest-876872140 tempest-VolumeQuotaClassesTest-876872140-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '196c2b91af4e4c588238489784255907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83f83d3ce2934cbdbb5d680757391e81', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:03.987705 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c849f7-7c77-4779-8b09-215471ec2b90 tempest-VolumeQuotaClassesTest-876872140 tempest-VolumeQuotaClassesTest-876872140-project-admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:03.988162 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c849f7-7c77-4779-8b09-215471ec2b90 tempest-VolumeQuotaClassesTest-876872140 tempest-VolumeQuotaClassesTest-876872140-project-admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:03.988638 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c849f7-7c77-4779-8b09-215471ec2b90 tempest-VolumeQuotaClassesTest-876872140 tempest-VolumeQuotaClassesTest-876872140-project-admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:03.989635 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c849f7-7c77-4779-8b09-215471ec2b90 tempest-VolumeQuotaClassesTest-876872140 tempest-VolumeQuotaClassesTest-876872140-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:15:03.990092 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c849f7-7c77-4779-8b09-215471ec2b90 tempest-VolumeQuotaClassesTest-876872140 tempest-VolumeQuotaClassesTest-876872140-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:03.992120 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c849f7-7c77-4779-8b09-215471ec2b90 tempest-VolumeQuotaClassesTest-876872140 tempest-VolumeQuotaClassesTest-876872140-project-admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:04.035675 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 891/1755] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:15:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:15:04.040613 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c849f7-7c77-4779-8b09-215471ec2b90 tempest-VolumeQuotaClassesTest-876872140 tempest-VolumeQuotaClassesTest-876872140-project-admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:04.045772 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-79c849f7-7c77-4779-8b09-215471ec2b90 tempest-VolumeQuotaClassesTest-876872140 tempest-VolumeQuotaClassesTest-876872140-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:04.080020 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c849f7-7c77-4779-8b09-215471ec2b90 tempest-VolumeQuotaClassesTest-876872140 tempest-VolumeQuotaClassesTest-876872140-project-admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:04.098763 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613d010c-08b2-4ddf-b538-55a069895527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:04.100015 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d010c-08b2-4ddf-b538-55a069895527 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:04.100578 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d010c-08b2-4ddf-b538-55a069895527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:04.101754 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613d010c-08b2-4ddf-b538-55a069895527 admin admin] PATH_INFO: `/v3/projects/f0964b39386343c5997a50981be53341` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:04.103601 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613d010c-08b2-4ddf-b538-55a069895527 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0964b39386343c5997a50981be53341)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:04.111304 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613d010c-08b2-4ddf-b538-55a069895527 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:04.155950 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ed2c7-8257-476f-89ce-6890010ee4ef admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:04.167731 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d968e9-217a-46eb-9d4f-70bf261533ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '451a9232f95e494683d81b6078a318d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:04.168765 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d968e9-217a-46eb-9d4f-70bf261533ba service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:04.169210 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d968e9-217a-46eb-9d4f-70bf261533ba service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:04.169642 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d968e9-217a-46eb-9d4f-70bf261533ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:04.219207 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:04.264468 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-613d010c-08b2-4ddf-b538-55a069895527 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0964b39386343c5997a50981be53341', 'request_id': 'req-613d010c-08b2-4ddf-b538-55a069895527'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:04.277205 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-613d010c-08b2-4ddf-b538-55a069895527 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0964b39386343c5997a50981be53341', 'request_id': 'req-613d010c-08b2-4ddf-b538-55a069895527'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:04.279654 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d968e9-217a-46eb-9d4f-70bf261533ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:04.281815 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d968e9-217a-46eb-9d4f-70bf261533ba service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:04.287028 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-613d010c-08b2-4ddf-b538-55a069895527 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0964b39386343c5997a50981be53341', 'request_id': 'req-613d010c-08b2-4ddf-b538-55a069895527'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:04.298269 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 865/1756] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:04.299754 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d968e9-217a-46eb-9d4f-70bf261533ba service nova] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:04.315230 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d968e9-217a-46eb-9d4f-70bf261533ba service nova] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:04.318537 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-613d010c-08b2-4ddf-b538-55a069895527 admin admin] The token cache is being invalidate because project f0964b39386343c5997a50981be53341 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:15:04.319043 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-613d010c-08b2-4ddf-b538-55a069895527 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:04.321285 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-613d010c-08b2-4ddf-b538-55a069895527 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:04.330647 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 892/1757] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:15:04 2022] DELETE /identity/v3/projects/f0964b39386343c5997a50981be53341 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:04.336529 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a669dc52-0b3c-48d5-957d-77c20aaf901e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:04.337059 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a669dc52-0b3c-48d5-957d-77c20aaf901e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:04.337500 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a669dc52-0b3c-48d5-957d-77c20aaf901e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:04.393645 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a669dc52-0b3c-48d5-957d-77c20aaf901e None None] MFA Rules not processed for user `387048f42b634693a7e827a8d88ef89b`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:04.395520 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a669dc52-0b3c-48d5-957d-77c20aaf901e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:04.528652 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 893/1758] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:04.530484 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a669dc52-0b3c-48d5-957d-77c20aaf901e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:04.532115 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a669dc52-0b3c-48d5-957d-77c20aaf901e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:04.575016 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0743133-3dea-4b14-9432-862a16a14875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:04.581180 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0743133-3dea-4b14-9432-862a16a14875 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:04.581483 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0743133-3dea-4b14-9432-862a16a14875 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:04.581753 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0743133-3dea-4b14-9432-862a16a14875 admin admin] PATH_INFO: `/v3/projects/d7439103a9514268ab6be38eb1860b7f/users/b179bcde825a465ab113c2fe2bc62661/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:04.596261 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0743133-3dea-4b14-9432-862a16a14875 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7439103a9514268ab6be38eb1860b7f, user_id=b179bcde825a465ab113c2fe2bc62661, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:04.601408 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0743133-3dea-4b14-9432-862a16a14875 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:04.605998 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:04.663848 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 866/1759] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:04 2022] PUT /identity/v3/projects/d7439103a9514268ab6be38eb1860b7f/users/b179bcde825a465ab113c2fe2bc62661/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:04.668163 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0743133-3dea-4b14-9432-862a16a14875 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:04.672703 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a0743133-3dea-4b14-9432-862a16a14875 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:04.698249 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a669dc52-0b3c-48d5-957d-77c20aaf901e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:04.851069 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0743133-3dea-4b14-9432-862a16a14875 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:04.883378 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349abf26-8915-4ae2-a012-d7abc986e0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:04.884581 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349abf26-8915-4ae2-a012-d7abc986e0ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:04.885274 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349abf26-8915-4ae2-a012-d7abc986e0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:04.885775 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349abf26-8915-4ae2-a012-d7abc986e0ed admin admin] PATH_INFO: `/v3/users/9322200d30be4cad976ebc35009d0572` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:04.888970 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349abf26-8915-4ae2-a012-d7abc986e0ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=9322200d30be4cad976ebc35009d0572)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:04.891363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349abf26-8915-4ae2-a012-d7abc986e0ed admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:04.914883 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114a931d-194c-4176-974a-77349f08266d tempest-VolumeQuotaClassesTest-876872140 tempest-VolumeQuotaClassesTest-876872140-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '196c2b91af4e4c588238489784255907', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '83f83d3ce2934cbdbb5d680757391e81', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:04.916157 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a931d-194c-4176-974a-77349f08266d tempest-VolumeQuotaClassesTest-876872140 tempest-VolumeQuotaClassesTest-876872140-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:04.916404 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a931d-194c-4176-974a-77349f08266d tempest-VolumeQuotaClassesTest-876872140 tempest-VolumeQuotaClassesTest-876872140-project-admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:04.916611 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114a931d-194c-4176-974a-77349f08266d tempest-VolumeQuotaClassesTest-876872140 tempest-VolumeQuotaClassesTest-876872140-project-admin] PATH_INFO: `/v3/projects/0e09f130f00b4cdbab24fb59123ef8d8` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:04.918046 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114a931d-194c-4176-974a-77349f08266d tempest-VolumeQuotaClassesTest-876872140 tempest-VolumeQuotaClassesTest-876872140-project-admin] RBAC: Authorizing `identity:delete_project(project_id=0e09f130f00b4cdbab24fb59123ef8d8)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:04.928432 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114a931d-194c-4176-974a-77349f08266d tempest-VolumeQuotaClassesTest-876872140 tempest-VolumeQuotaClassesTest-876872140-project-admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:04.959188 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:05.025292 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-114a931d-194c-4176-974a-77349f08266d tempest-VolumeQuotaClassesTest-876872140 tempest-VolumeQuotaClassesTest-876872140-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e09f130f00b4cdbab24fb59123ef8d8', 'request_id': 'req-114a931d-194c-4176-974a-77349f08266d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:05.030288 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-349abf26-8915-4ae2-a012-d7abc986e0ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9322200d30be4cad976ebc35009d0572'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:05.043215 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-114a931d-194c-4176-974a-77349f08266d tempest-VolumeQuotaClassesTest-876872140 tempest-VolumeQuotaClassesTest-876872140-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e09f130f00b4cdbab24fb59123ef8d8', 'request_id': 'req-114a931d-194c-4176-974a-77349f08266d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:05.053907 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-349abf26-8915-4ae2-a012-d7abc986e0ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9322200d30be4cad976ebc35009d0572'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:05.056368 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-114a931d-194c-4176-974a-77349f08266d tempest-VolumeQuotaClassesTest-876872140 tempest-VolumeQuotaClassesTest-876872140-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e09f130f00b4cdbab24fb59123ef8d8', 'request_id': 'req-114a931d-194c-4176-974a-77349f08266d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:05.068038 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-349abf26-8915-4ae2-a012-d7abc986e0ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9322200d30be4cad976ebc35009d0572'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:05.068793 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-349abf26-8915-4ae2-a012-d7abc986e0ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9322200d30be4cad976ebc35009d0572'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:05.070605 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-349abf26-8915-4ae2-a012-d7abc986e0ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9322200d30be4cad976ebc35009d0572'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:05.082685 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-114a931d-194c-4176-974a-77349f08266d tempest-VolumeQuotaClassesTest-876872140 tempest-VolumeQuotaClassesTest-876872140-project-admin] The token cache is being invalidate because project 0e09f130f00b4cdbab24fb59123ef8d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:15:05.083287 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-114a931d-194c-4176-974a-77349f08266d tempest-VolumeQuotaClassesTest-876872140 tempest-VolumeQuotaClassesTest-876872140-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:05.086192 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-114a931d-194c-4176-974a-77349f08266d tempest-VolumeQuotaClassesTest-876872140 tempest-VolumeQuotaClassesTest-876872140-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:05.089319 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 894/1760] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:15:04 2022] DELETE /identity/v3/projects/0e09f130f00b4cdbab24fb59123ef8d8 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:05.093162 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d130d90-867a-44ac-b1ac-dca767ecad0e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:05.094496 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d130d90-867a-44ac-b1ac-dca767ecad0e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:05.095133 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d130d90-867a-44ac-b1ac-dca767ecad0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:05.109303 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 867/1761] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:15:04 2022] DELETE /identity/v3/users/9322200d30be4cad976ebc35009d0572 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:05.110060 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349abf26-8915-4ae2-a012-d7abc986e0ed admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:05.112508 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-349abf26-8915-4ae2-a012-d7abc986e0ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:05.160953 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7d130d90-867a-44ac-b1ac-dca767ecad0e None None] MFA Rules not processed for user `b179bcde825a465ab113c2fe2bc62661`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:05.162296 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7d130d90-867a-44ac-b1ac-dca767ecad0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:05.276864 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349abf26-8915-4ae2-a012-d7abc986e0ed admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:05.297640 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3519743-8d33-4353-860d-1b5635b092c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:05.299555 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3519743-8d33-4353-860d-1b5635b092c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:05.300531 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3519743-8d33-4353-860d-1b5635b092c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:05.300864 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3519743-8d33-4353-860d-1b5635b092c0 admin admin] PATH_INFO: `/v3/users/61dd84b3080d445abb173ab3642f6886` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:05.302818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3519743-8d33-4353-860d-1b5635b092c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61dd84b3080d445abb173ab3642f6886)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:05.306098 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3519743-8d33-4353-860d-1b5635b092c0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:05.342468 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 895/1762] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:05 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:05.358753 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d130d90-867a-44ac-b1ac-dca767ecad0e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:05.359083 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:INSERT=2,keystone:SELECT=87,keystone:DELETE=11 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:05.363486 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7d130d90-867a-44ac-b1ac-dca767ecad0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:05.462707 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d130d90-867a-44ac-b1ac-dca767ecad0e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:05.468102 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20290d7c-30a3-4ab2-88ca-0739ea4c135d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:05.468844 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20290d7c-30a3-4ab2-88ca-0739ea4c135d service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:05.469329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20290d7c-30a3-4ab2-88ca-0739ea4c135d service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:05.469710 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20290d7c-30a3-4ab2-88ca-0739ea4c135d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:05.470092 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e3519743-8d33-4353-860d-1b5635b092c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61dd84b3080d445abb173ab3642f6886'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:05.481817 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20290d7c-30a3-4ab2-88ca-0739ea4c135d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:05.484295 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20290d7c-30a3-4ab2-88ca-0739ea4c135d service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:05.499273 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e3519743-8d33-4353-860d-1b5635b092c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61dd84b3080d445abb173ab3642f6886'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:05.508074 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e3519743-8d33-4353-860d-1b5635b092c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61dd84b3080d445abb173ab3642f6886'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:05.508847 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e3519743-8d33-4353-860d-1b5635b092c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61dd84b3080d445abb173ab3642f6886'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:05.509492 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e3519743-8d33-4353-860d-1b5635b092c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61dd84b3080d445abb173ab3642f6886'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:05.517047 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 896/1763] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:05 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:05.518020 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20290d7c-30a3-4ab2-88ca-0739ea4c135d service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:05.519596 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-20290d7c-30a3-4ab2-88ca-0739ea4c135d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:05.536123 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 868/1764] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:15:05 2022] DELETE /identity/v3/users/61dd84b3080d445abb173ab3642f6886 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:05.538377 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197fbc07-b146-4de4-8fef-9d1e9c782d4a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:05.538641 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197fbc07-b146-4de4-8fef-9d1e9c782d4a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:05.538952 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197fbc07-b146-4de4-8fef-9d1e9c782d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:05.541795 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20290d7c-30a3-4ab2-88ca-0739ea4c135d service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:05.565259 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-197fbc07-b146-4de4-8fef-9d1e9c782d4a None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:05.567350 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-197fbc07-b146-4de4-8fef-9d1e9c782d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:05.623493 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:05.704204 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d77be00-1c09-4f7e-bd3f-d0ae35b5c89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:05.705970 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77be00-1c09-4f7e-bd3f-d0ae35b5c89d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:05.706474 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77be00-1c09-4f7e-bd3f-d0ae35b5c89d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:05.706982 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77be00-1c09-4f7e-bd3f-d0ae35b5c89d admin admin] PATH_INFO: `/v3/users/9f027a39f90a469bb21dd585fc02ab4c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:05.720186 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d77be00-1c09-4f7e-bd3f-d0ae35b5c89d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f027a39f90a469bb21dd585fc02ab4c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:05.722929 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d77be00-1c09-4f7e-bd3f-d0ae35b5c89d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:05.724729 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 869/1765] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:05.725935 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197fbc07-b146-4de4-8fef-9d1e9c782d4a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:05.736164 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-197fbc07-b146-4de4-8fef-9d1e9c782d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:05.826455 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197fbc07-b146-4de4-8fef-9d1e9c782d4a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:05.832003 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edcc3b7-00a3-431b-8917-418750026181 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:05.832268 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6d77be00-1c09-4f7e-bd3f-d0ae35b5c89d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f027a39f90a469bb21dd585fc02ab4c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:05.832904 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edcc3b7-00a3-431b-8917-418750026181 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:05.833372 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edcc3b7-00a3-431b-8917-418750026181 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:05.833725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edcc3b7-00a3-431b-8917-418750026181 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:05.841070 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7edcc3b7-00a3-431b-8917-418750026181 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:05.844765 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6d77be00-1c09-4f7e-bd3f-d0ae35b5c89d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f027a39f90a469bb21dd585fc02ab4c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:05.845555 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6d77be00-1c09-4f7e-bd3f-d0ae35b5c89d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f027a39f90a469bb21dd585fc02ab4c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:05.858203 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6d77be00-1c09-4f7e-bd3f-d0ae35b5c89d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f027a39f90a469bb21dd585fc02ab4c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:05.859175 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6d77be00-1c09-4f7e-bd3f-d0ae35b5c89d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f027a39f90a469bb21dd585fc02ab4c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:05.913928 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 897/1766] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:15:05 2022] DELETE /identity/v3/users/9f027a39f90a469bb21dd585fc02ab4c => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:05.914701 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d77be00-1c09-4f7e-bd3f-d0ae35b5c89d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:05.916599 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6d77be00-1c09-4f7e-bd3f-d0ae35b5c89d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:05.972483 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:06.006212 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edcc3b7-00a3-431b-8917-418750026181 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:06.009280 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edcc3b7-00a3-431b-8917-418750026181 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:06.044474 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 870/1767] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:05 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:06.045423 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edcc3b7-00a3-431b-8917-418750026181 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:06.053991 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7edcc3b7-00a3-431b-8917-418750026181 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:06.101295 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d77be00-1c09-4f7e-bd3f-d0ae35b5c89d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:06.125732 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba71d62-84df-4cdb-90dc-deeb6cfa4063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:06.126655 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba71d62-84df-4cdb-90dc-deeb6cfa4063 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:06.127002 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba71d62-84df-4cdb-90dc-deeb6cfa4063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:06.127291 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba71d62-84df-4cdb-90dc-deeb6cfa4063 admin admin] PATH_INFO: `/v3/projects/d972ac5aa1094b618977c30771245f6e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:06.139834 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba71d62-84df-4cdb-90dc-deeb6cfa4063 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d972ac5aa1094b618977c30771245f6e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:06.141815 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba71d62-84df-4cdb-90dc-deeb6cfa4063 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:06.163473 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edcc3b7-00a3-431b-8917-418750026181 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:06.246907 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bba71d62-84df-4cdb-90dc-deeb6cfa4063 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd972ac5aa1094b618977c30771245f6e', 'request_id': 'req-bba71d62-84df-4cdb-90dc-deeb6cfa4063'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:06.248732 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e15b36-8d43-4b93-8272-eeb630109f12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:06.249599 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e15b36-8d43-4b93-8272-eeb630109f12 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:06.250067 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e15b36-8d43-4b93-8272-eeb630109f12 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:06.250447 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e15b36-8d43-4b93-8272-eeb630109f12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:06.253348 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:06.256149 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-35e15b36-8d43-4b93-8272-eeb630109f12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:06.274930 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bba71d62-84df-4cdb-90dc-deeb6cfa4063 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd972ac5aa1094b618977c30771245f6e', 'request_id': 'req-bba71d62-84df-4cdb-90dc-deeb6cfa4063'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:06.286909 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bba71d62-84df-4cdb-90dc-deeb6cfa4063 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd972ac5aa1094b618977c30771245f6e', 'request_id': 'req-bba71d62-84df-4cdb-90dc-deeb6cfa4063'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:06.287165 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e15b36-8d43-4b93-8272-eeb630109f12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:06.289807 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e15b36-8d43-4b93-8272-eeb630109f12 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:06.312029 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 871/1768] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:06.316647 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d2aff-ed31-42dc-9e3d-48624bc8c0b9 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:06.327097 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bba71d62-84df-4cdb-90dc-deeb6cfa4063 admin admin] The token cache is being invalidate because project d972ac5aa1094b618977c30771245f6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:15:06.327097 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bba71d62-84df-4cdb-90dc-deeb6cfa4063 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:06.327097 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bba71d62-84df-4cdb-90dc-deeb6cfa4063 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:06.330985 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 898/1769] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:15:05 2022] DELETE /identity/v3/projects/d972ac5aa1094b618977c30771245f6e => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:06.331234 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d2aff-ed31-42dc-9e3d-48624bc8c0b9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:06.331234 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d2aff-ed31-42dc-9e3d-48624bc8c0b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:06.333535 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba71d62-84df-4cdb-90dc-deeb6cfa4063 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:06.335605 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bba71d62-84df-4cdb-90dc-deeb6cfa4063 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:06.401944 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-cd2d2aff-ed31-42dc-9e3d-48624bc8c0b9 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:06.405959 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cd2d2aff-ed31-42dc-9e3d-48624bc8c0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:06.513865 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba71d62-84df-4cdb-90dc-deeb6cfa4063 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:06.529407 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7328587c-78d2-40f6-879c-dddaf2ddc0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:06.531539 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7328587c-78d2-40f6-879c-dddaf2ddc0f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:06.533209 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7328587c-78d2-40f6-879c-dddaf2ddc0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:06.534404 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7328587c-78d2-40f6-879c-dddaf2ddc0f0 admin admin] PATH_INFO: `/v3/users/096845fd015e44f68c14b740d386cc63` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:06.538270 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7328587c-78d2-40f6-879c-dddaf2ddc0f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=096845fd015e44f68c14b740d386cc63)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:06.541133 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 872/1770] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:06.546839 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7328587c-78d2-40f6-879c-dddaf2ddc0f0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:06.556338 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2d2aff-ed31-42dc-9e3d-48624bc8c0b9 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:06.558270 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cd2d2aff-ed31-42dc-9e3d-48624bc8c0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:06.665211 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:06.666505 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7328587c-78d2-40f6-879c-dddaf2ddc0f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '096845fd015e44f68c14b740d386cc63'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:06.675975 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2d2aff-ed31-42dc-9e3d-48624bc8c0b9 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:06.682610 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7328587c-78d2-40f6-879c-dddaf2ddc0f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '096845fd015e44f68c14b740d386cc63'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:06.684048 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7328587c-78d2-40f6-879c-dddaf2ddc0f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '096845fd015e44f68c14b740d386cc63'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:06.684962 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7539a3ce-7acf-4a6f-8ffc-d42bdf4e2238 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:06.685996 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7539a3ce-7acf-4a6f-8ffc-d42bdf4e2238 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:06.686478 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7539a3ce-7acf-4a6f-8ffc-d42bdf4e2238 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:06.687067 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7539a3ce-7acf-4a6f-8ffc-d42bdf4e2238 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:06.700624 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7539a3ce-7acf-4a6f-8ffc-d42bdf4e2238 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:06.700783 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7328587c-78d2-40f6-879c-dddaf2ddc0f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '096845fd015e44f68c14b740d386cc63'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:06.701623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7328587c-78d2-40f6-879c-dddaf2ddc0f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '096845fd015e44f68c14b740d386cc63'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:06.703705 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7539a3ce-7acf-4a6f-8ffc-d42bdf4e2238 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:06.723928 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 873/1771] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:15:06.733902 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 899/1772] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:15:06 2022] DELETE /identity/v3/users/096845fd015e44f68c14b740d386cc63 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:06.847529 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607fd67e-8788-4c22-b989-0b5d8e331a7e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:06.847847 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607fd67e-8788-4c22-b989-0b5d8e331a7e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:06.848130 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607fd67e-8788-4c22-b989-0b5d8e331a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:06.881838 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-607fd67e-8788-4c22-b989-0b5d8e331a7e None None] MFA Rules not processed for user `5a990918de6d409a949616da90de6a5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:06.883238 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-607fd67e-8788-4c22-b989-0b5d8e331a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:06.944706 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7328587c-78d2-40f6-879c-dddaf2ddc0f0 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:06.946239 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7328587c-78d2-40f6-879c-dddaf2ddc0f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:06.998908 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 874/1773] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:07.090592 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7328587c-78d2-40f6-879c-dddaf2ddc0f0 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:07.097555 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d674870-ca96-4835-8d1b-07b454516eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:07.099298 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d674870-ca96-4835-8d1b-07b454516eec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:07.109121 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d674870-ca96-4835-8d1b-07b454516eec admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:07.109698 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d674870-ca96-4835-8d1b-07b454516eec admin admin] PATH_INFO: `/v3/projects/8293dd408a05464f987aa79eed0d09fa` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:07.131551 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d674870-ca96-4835-8d1b-07b454516eec admin admin] RBAC: Authorizing `identity:delete_project(project_id=8293dd408a05464f987aa79eed0d09fa)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:07.137197 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d674870-ca96-4835-8d1b-07b454516eec admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:07.144332 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:07.236908 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7d674870-ca96-4835-8d1b-07b454516eec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8293dd408a05464f987aa79eed0d09fa', 'request_id': 'req-7d674870-ca96-4835-8d1b-07b454516eec'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:07.248135 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7d674870-ca96-4835-8d1b-07b454516eec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8293dd408a05464f987aa79eed0d09fa', 'request_id': 'req-7d674870-ca96-4835-8d1b-07b454516eec'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:07.259348 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7d674870-ca96-4835-8d1b-07b454516eec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8293dd408a05464f987aa79eed0d09fa', 'request_id': 'req-7d674870-ca96-4835-8d1b-07b454516eec'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:07.291824 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7d674870-ca96-4835-8d1b-07b454516eec admin admin] The token cache is being invalidate because project 8293dd408a05464f987aa79eed0d09fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:15:07.292108 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7d674870-ca96-4835-8d1b-07b454516eec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:07.298919 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7d674870-ca96-4835-8d1b-07b454516eec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:07.300899 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 900/1774] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:15:06 2022] DELETE /identity/v3/projects/8293dd408a05464f987aa79eed0d09fa => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:15:07.307846 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b995f2ea-222d-492f-9e3d-39f9fb58e9dd None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:07.309039 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b995f2ea-222d-492f-9e3d-39f9fb58e9dd None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:07.309487 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b995f2ea-222d-492f-9e3d-39f9fb58e9dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:07.321421 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d674870-ca96-4835-8d1b-07b454516eec admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:07.323222 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7d674870-ca96-4835-8d1b-07b454516eec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:07.337722 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b995f2ea-222d-492f-9e3d-39f9fb58e9dd None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:07.338982 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b995f2ea-222d-492f-9e3d-39f9fb58e9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:07.410984 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 875/1775] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:07.419553 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f69d8-04cd-4610-800f-596f6500bd1d None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:07.419996 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f69d8-04cd-4610-800f-596f6500bd1d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:07.420385 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f69d8-04cd-4610-800f-596f6500bd1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:07.447600 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b25f69d8-04cd-4610-800f-596f6500bd1d None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:07.447790 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d674870-ca96-4835-8d1b-07b454516eec admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:07.449421 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b25f69d8-04cd-4610-800f-596f6500bd1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:07.452698 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d522f86-5abe-4b24-afed-7e2d931e6862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:07.453666 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d522f86-5abe-4b24-afed-7e2d931e6862 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:07.454009 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d522f86-5abe-4b24-afed-7e2d931e6862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:07.454447 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d522f86-5abe-4b24-afed-7e2d931e6862 admin admin] PATH_INFO: `/v3/projects/763b86e473ee403091dca92587e2f7e9` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:07.456681 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d522f86-5abe-4b24-afed-7e2d931e6862 admin admin] RBAC: Authorizing `identity:delete_project(project_id=763b86e473ee403091dca92587e2f7e9)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:07.458681 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d522f86-5abe-4b24-afed-7e2d931e6862 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:07.461400 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 876/1776] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:07.467145 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f69d8-04cd-4610-800f-596f6500bd1d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:07.479529 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f69d8-04cd-4610-800f-596f6500bd1d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:07.488379 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09abc1c3-fde6-48e4-9e23-8e6c1719251d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:07.489484 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09abc1c3-fde6-48e4-9e23-8e6c1719251d admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:07.490233 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09abc1c3-fde6-48e4-9e23-8e6c1719251d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:07.491577 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09abc1c3-fde6-48e4-9e23-8e6c1719251d admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:07.492603 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09abc1c3-fde6-48e4-9e23-8e6c1719251d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:15:07.493036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09abc1c3-fde6-48e4-9e23-8e6c1719251d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:07.494989 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09abc1c3-fde6-48e4-9e23-8e6c1719251d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:07.520700 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 877/1777] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:15:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:15:07.525821 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09abc1c3-fde6-48e4-9e23-8e6c1719251d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:07.528299 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-09abc1c3-fde6-48e4-9e23-8e6c1719251d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:07.544394 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09abc1c3-fde6-48e4-9e23-8e6c1719251d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:07.555358 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1d522f86-5abe-4b24-afed-7e2d931e6862 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '763b86e473ee403091dca92587e2f7e9', 'request_id': 'req-1d522f86-5abe-4b24-afed-7e2d931e6862'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:07.570791 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1d522f86-5abe-4b24-afed-7e2d931e6862 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '763b86e473ee403091dca92587e2f7e9', 'request_id': 'req-1d522f86-5abe-4b24-afed-7e2d931e6862'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:07.598667 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1d522f86-5abe-4b24-afed-7e2d931e6862 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '763b86e473ee403091dca92587e2f7e9', 'request_id': 'req-1d522f86-5abe-4b24-afed-7e2d931e6862'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:07.620224 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1d522f86-5abe-4b24-afed-7e2d931e6862 admin admin] The token cache is being invalidate because project 763b86e473ee403091dca92587e2f7e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:15:07.621452 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1d522f86-5abe-4b24-afed-7e2d931e6862 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:07.622912 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1d522f86-5abe-4b24-afed-7e2d931e6862 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:07.626901 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 901/1778] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:15:07 2022] DELETE /identity/v3/projects/763b86e473ee403091dca92587e2f7e9 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:15:07.628165 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d522f86-5abe-4b24-afed-7e2d931e6862 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:07.630756 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1d522f86-5abe-4b24-afed-7e2d931e6862 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:07.670649 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:07.691353 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f696a2d9-4df0-481f-908f-775fc2bbec13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:07.692655 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f696a2d9-4df0-481f-908f-775fc2bbec13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:07.693261 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f696a2d9-4df0-481f-908f-775fc2bbec13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:07.693711 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f696a2d9-4df0-481f-908f-775fc2bbec13 admin admin] PATH_INFO: `/v3/projects/135d51d1a8be4ba593cc5eab07b74c68` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:07.696436 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f696a2d9-4df0-481f-908f-775fc2bbec13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=135d51d1a8be4ba593cc5eab07b74c68)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:07.698543 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f696a2d9-4df0-481f-908f-775fc2bbec13 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:07.733726 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:07.746469 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d522f86-5abe-4b24-afed-7e2d931e6862 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:07.757585 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73468e0b-5835-456b-8e24-183a9023f71f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:07.758647 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73468e0b-5835-456b-8e24-183a9023f71f admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:07.759033 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73468e0b-5835-456b-8e24-183a9023f71f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:07.759482 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73468e0b-5835-456b-8e24-183a9023f71f admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:07.760387 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73468e0b-5835-456b-8e24-183a9023f71f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:07.763751 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73468e0b-5835-456b-8e24-183a9023f71f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:07.786270 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f696a2d9-4df0-481f-908f-775fc2bbec13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '135d51d1a8be4ba593cc5eab07b74c68', 'request_id': 'req-f696a2d9-4df0-481f-908f-775fc2bbec13'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:07.794584 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f696a2d9-4df0-481f-908f-775fc2bbec13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '135d51d1a8be4ba593cc5eab07b74c68', 'request_id': 'req-f696a2d9-4df0-481f-908f-775fc2bbec13'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:07.806791 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f696a2d9-4df0-481f-908f-775fc2bbec13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '135d51d1a8be4ba593cc5eab07b74c68', 'request_id': 'req-f696a2d9-4df0-481f-908f-775fc2bbec13'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:07.828622 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 902/1779] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:15:07 2022] POST /identity/v3/projects => generated 365 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:07.832938 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73468e0b-5835-456b-8e24-183a9023f71f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:07.836622 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-73468e0b-5835-456b-8e24-183a9023f71f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:07.839993 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f696a2d9-4df0-481f-908f-775fc2bbec13 admin admin] The token cache is being invalidate because project 135d51d1a8be4ba593cc5eab07b74c68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:15:07.840530 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f696a2d9-4df0-481f-908f-775fc2bbec13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:07.841415 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-f696a2d9-4df0-481f-908f-775fc2bbec13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:07.849983 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 878/1780] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:15:07 2022] DELETE /identity/v3/projects/135d51d1a8be4ba593cc5eab07b74c68 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:07.851116 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f696a2d9-4df0-481f-908f-775fc2bbec13 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:07.856504 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f696a2d9-4df0-481f-908f-775fc2bbec13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:07.941526 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73468e0b-5835-456b-8e24-183a9023f71f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:07.946999 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb708cac-9fa6-4002-90f5-70584909d92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:07.947988 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb708cac-9fa6-4002-90f5-70584909d92a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:07.948407 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb708cac-9fa6-4002-90f5-70584909d92a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:07.949538 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb708cac-9fa6-4002-90f5-70584909d92a admin admin] PATH_INFO: `/v3/users/196c2b91af4e4c588238489784255907` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:07.951168 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb708cac-9fa6-4002-90f5-70584909d92a admin admin] RBAC: Authorizing `identity:delete_user(user_id=196c2b91af4e4c588238489784255907)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:07.954368 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb708cac-9fa6-4002-90f5-70584909d92a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:08.016846 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f696a2d9-4df0-481f-908f-775fc2bbec13 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:08.029396 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13fafb4-ec18-466c-87e5-1296a69c8058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:08.030305 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fafb4-ec18-466c-87e5-1296a69c8058 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:08.030744 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fafb4-ec18-466c-87e5-1296a69c8058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:08.031145 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fafb4-ec18-466c-87e5-1296a69c8058 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:08.031758 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13fafb4-ec18-466c-87e5-1296a69c8058 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:08.033717 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13fafb4-ec18-466c-87e5-1296a69c8058 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:08.040076 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fb708cac-9fa6-4002-90f5-70584909d92a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '196c2b91af4e4c588238489784255907'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:08.056154 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fb708cac-9fa6-4002-90f5-70584909d92a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '196c2b91af4e4c588238489784255907'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:08.056914 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fb708cac-9fa6-4002-90f5-70584909d92a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '196c2b91af4e4c588238489784255907'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:08.066741 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fb708cac-9fa6-4002-90f5-70584909d92a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '196c2b91af4e4c588238489784255907'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:08.068010 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fb708cac-9fa6-4002-90f5-70584909d92a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '196c2b91af4e4c588238489784255907'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:08.074594 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 879/1781] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:15:07 2022] POST /identity/v3/users => generated 283 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:08.079465 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d28b9d-16f0-4376-b52c-4e0799b12afa None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:08.079980 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d28b9d-16f0-4376-b52c-4e0799b12afa None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:08.080444 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d28b9d-16f0-4376-b52c-4e0799b12afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:08.112568 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 903/1782] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:15:07 2022] DELETE /identity/v3/users/196c2b91af4e4c588238489784255907 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:08.114097 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb708cac-9fa6-4002-90f5-70584909d92a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:08.117049 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fb708cac-9fa6-4002-90f5-70584909d92a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:08.130142 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-58d28b9d-16f0-4376-b52c-4e0799b12afa None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:08.133524 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-58d28b9d-16f0-4376-b52c-4e0799b12afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:08.192505 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:08.297734 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb708cac-9fa6-4002-90f5-70584909d92a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:08.308152 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 880/1783] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:08.308352 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec88728c-b91e-4326-8b15-3211788b0f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:08.310225 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04c441-3b07-4117-a404-b5f0d111a8e8 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:08.310943 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04c441-3b07-4117-a404-b5f0d111a8e8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:08.310943 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04c441-3b07-4117-a404-b5f0d111a8e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:08.314994 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88728c-b91e-4326-8b15-3211788b0f83 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:08.317124 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88728c-b91e-4326-8b15-3211788b0f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:08.317877 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88728c-b91e-4326-8b15-3211788b0f83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:08.321825 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec88728c-b91e-4326-8b15-3211788b0f83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:08.326046 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec88728c-b91e-4326-8b15-3211788b0f83 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:08.380171 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 904/1784] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:08.381771 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec88728c-b91e-4326-8b15-3211788b0f83 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:08.398619 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3e04c441-3b07-4117-a404-b5f0d111a8e8 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:08.403581 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3e04c441-3b07-4117-a404-b5f0d111a8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:08.415252 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec88728c-b91e-4326-8b15-3211788b0f83 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:08.422437 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 881/1785] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:08.424773 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a13dd6a-29c3-4535-bf39-903813c4ead9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:08.425867 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a13dd6a-29c3-4535-bf39-903813c4ead9 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:08.426165 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a13dd6a-29c3-4535-bf39-903813c4ead9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:08.426536 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a13dd6a-29c3-4535-bf39-903813c4ead9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:08.427374 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a13dd6a-29c3-4535-bf39-903813c4ead9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:08.427579 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e04c441-3b07-4117-a404-b5f0d111a8e8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:08.429314 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a13dd6a-29c3-4535-bf39-903813c4ead9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:08.452214 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e04c441-3b07-4117-a404-b5f0d111a8e8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:08.458883 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 905/1786] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:08.460169 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a13dd6a-29c3-4535-bf39-903813c4ead9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:08.466348 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e476c1b-f559-47b3-83de-c151d81614a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:08.467556 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e476c1b-f559-47b3-83de-c151d81614a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:08.468375 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e476c1b-f559-47b3-83de-c151d81614a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:08.468899 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e476c1b-f559-47b3-83de-c151d81614a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:08.470953 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e476c1b-f559-47b3-83de-c151d81614a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:15:08.471967 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e476c1b-f559-47b3-83de-c151d81614a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:08.474922 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e476c1b-f559-47b3-83de-c151d81614a8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:08.492222 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a13dd6a-29c3-4535-bf39-903813c4ead9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:08.493759 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 882/1787] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:15:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:15:08.498007 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e476c1b-f559-47b3-83de-c151d81614a8 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:08.499703 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc30118d-d764-469c-bffa-cd92a2250e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:08.509420 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc30118d-d764-469c-bffa-cd92a2250e7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:08.509769 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc30118d-d764-469c-bffa-cd92a2250e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:08.510107 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc30118d-d764-469c-bffa-cd92a2250e7a admin admin] PATH_INFO: `/v3/projects/aae2b9395d3a48e4be2c252e2ae938db/users/8f512ee1b62147c4927fc0d92a029ebb/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:08.513776 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e476c1b-f559-47b3-83de-c151d81614a8 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:08.520921 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad4197d-ba2f-467c-9335-aa42e663d89b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:08.522342 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad4197d-ba2f-467c-9335-aa42e663d89b admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:08.522676 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad4197d-ba2f-467c-9335-aa42e663d89b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:08.523023 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad4197d-ba2f-467c-9335-aa42e663d89b admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:08.524082 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad4197d-ba2f-467c-9335-aa42e663d89b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:08.527369 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad4197d-ba2f-467c-9335-aa42e663d89b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:08.530793 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc30118d-d764-469c-bffa-cd92a2250e7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=aae2b9395d3a48e4be2c252e2ae938db, user_id=8f512ee1b62147c4927fc0d92a029ebb, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:08.533654 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc30118d-d764-469c-bffa-cd92a2250e7a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:08.555745 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 906/1788] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:08 2022] PUT /identity/v3/projects/aae2b9395d3a48e4be2c252e2ae938db/users/8f512ee1b62147c4927fc0d92a029ebb/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:08.567577 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:08.569846 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232d60e-fdd6-4745-862d-628982751e16 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:08.570166 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232d60e-fdd6-4745-862d-628982751e16 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:08.570968 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e232d60e-fdd6-4745-862d-628982751e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:08.591182 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 883/1789] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:15:08 2022] POST /identity/v3/projects => generated 381 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:08.593579 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad4197d-ba2f-467c-9335-aa42e663d89b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:08.607366 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e232d60e-fdd6-4745-862d-628982751e16 None None] MFA Rules not processed for user `8f512ee1b62147c4927fc0d92a029ebb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:08.608771 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e232d60e-fdd6-4745-862d-628982751e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:08.679359 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad4197d-ba2f-467c-9335-aa42e663d89b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:08.686819 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e968c096-7e85-463b-9c97-c37771245a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:08.687556 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e968c096-7e85-463b-9c97-c37771245a40 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:08.688014 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e968c096-7e85-463b-9c97-c37771245a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:08.688306 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e968c096-7e85-463b-9c97-c37771245a40 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:08.689017 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e968c096-7e85-463b-9c97-c37771245a40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:08.691153 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e968c096-7e85-463b-9c97-c37771245a40 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:08.700361 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 907/1790] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:08 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:15:08.705186 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e232d60e-fdd6-4745-862d-628982751e16 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:08.720206 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e232d60e-fdd6-4745-862d-628982751e16 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:08.720959 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 884/1791] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:15:08 2022] POST /identity/v3/users => generated 291 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:08.728041 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e968c096-7e85-463b-9c97-c37771245a40 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:08.731169 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e968c096-7e85-463b-9c97-c37771245a40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:08.742576 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e5435-5644-4162-8dc8-b40c48ed48e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:08.743380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e5435-5644-4162-8dc8-b40c48ed48e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:08.743637 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e5435-5644-4162-8dc8-b40c48ed48e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:08.743935 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e5435-5644-4162-8dc8-b40c48ed48e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:08.744615 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e968c096-7e85-463b-9c97-c37771245a40 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:08.744722 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724e5435-5644-4162-8dc8-b40c48ed48e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:08.747488 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724e5435-5644-4162-8dc8-b40c48ed48e2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:08.750597 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef01521a-c013-4676-ab74-4cd263464d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:08.751767 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01521a-c013-4676-ab74-4cd263464d7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:08.752134 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01521a-c013-4676-ab74-4cd263464d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:08.752522 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01521a-c013-4676-ab74-4cd263464d7f admin admin] PATH_INFO: `/v3/projects/83f83d3ce2934cbdbb5d680757391e81` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:08.753888 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef01521a-c013-4676-ab74-4cd263464d7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=83f83d3ce2934cbdbb5d680757391e81)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:08.755875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef01521a-c013-4676-ab74-4cd263464d7f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:08.779243 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 908/1792] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:15:08 2022] POST /identity/v3/projects => generated 363 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:08.780309 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e5435-5644-4162-8dc8-b40c48ed48e2 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:08.810935 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ef01521a-c013-4676-ab74-4cd263464d7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83f83d3ce2934cbdbb5d680757391e81', 'request_id': 'req-ef01521a-c013-4676-ab74-4cd263464d7f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:08.816890 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ef01521a-c013-4676-ab74-4cd263464d7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83f83d3ce2934cbdbb5d680757391e81', 'request_id': 'req-ef01521a-c013-4676-ab74-4cd263464d7f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:08.823592 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ef01521a-c013-4676-ab74-4cd263464d7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83f83d3ce2934cbdbb5d680757391e81', 'request_id': 'req-ef01521a-c013-4676-ab74-4cd263464d7f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:08.837377 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ef01521a-c013-4676-ab74-4cd263464d7f admin admin] The token cache is being invalidate because project 83f83d3ce2934cbdbb5d680757391e81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:15:08.837668 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ef01521a-c013-4676-ab74-4cd263464d7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:08.838311 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ef01521a-c013-4676-ab74-4cd263464d7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:08.839388 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:08.840313 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 885/1793] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:15:08 2022] DELETE /identity/v3/projects/83f83d3ce2934cbdbb5d680757391e81 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:08.841170 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef01521a-c013-4676-ab74-4cd263464d7f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:08.842671 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ef01521a-c013-4676-ab74-4cd263464d7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:08.852788 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e5435-5644-4162-8dc8-b40c48ed48e2 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:08.907189 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef01521a-c013-4676-ab74-4cd263464d7f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:08.910598 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b31915-5d74-457e-badc-d271aa234d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:08.911305 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b31915-5d74-457e-badc-d271aa234d60 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:08.911591 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b31915-5d74-457e-badc-d271aa234d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:08.911808 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2c4057-d989-4e97-b5db-e11243c91ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:08.911939 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b31915-5d74-457e-badc-d271aa234d60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:08.912473 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c4057-d989-4e97-b5db-e11243c91ee6 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:08.912758 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c4057-d989-4e97-b5db-e11243c91ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:08.912963 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b31915-5d74-457e-badc-d271aa234d60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:08.913061 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2c4057-d989-4e97-b5db-e11243c91ee6 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:08.913719 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2c4057-d989-4e97-b5db-e11243c91ee6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:08.914565 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b31915-5d74-457e-badc-d271aa234d60 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:08.915649 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2c4057-d989-4e97-b5db-e11243c91ee6 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:08.928464 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 909/1794] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:08.928464 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d3831e-23ff-44c6-af2a-427055f0026a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:08.928464 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d3831e-23ff-44c6-af2a-427055f0026a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:08.928464 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d3831e-23ff-44c6-af2a-427055f0026a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:08.931104 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:INSERT=8,keystone:SELECT=85,keystone:DELETE=7 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:08.940979 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 886/1795] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:15:08 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:08.942153 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2c4057-d989-4e97-b5db-e11243c91ee6 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:08.945851 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6e2c4057-d989-4e97-b5db-e11243c91ee6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:08.948644 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-53d3831e-23ff-44c6-af2a-427055f0026a None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:08.949801 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-53d3831e-23ff-44c6-af2a-427055f0026a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:08.957713 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 910/1796] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:08.958140 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2c4057-d989-4e97-b5db-e11243c91ee6 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:08.958496 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d3831e-23ff-44c6-af2a-427055f0026a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:08.967023 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c65db57-887a-4d5c-a9d5-cc7120c0b99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:08.967695 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c65db57-887a-4d5c-a9d5-cc7120c0b99a admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:08.968230 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c65db57-887a-4d5c-a9d5-cc7120c0b99a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:08.968510 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c65db57-887a-4d5c-a9d5-cc7120c0b99a admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:08.969269 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c65db57-887a-4d5c-a9d5-cc7120c0b99a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:08.970920 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c65db57-887a-4d5c-a9d5-cc7120c0b99a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:08.973441 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d3831e-23ff-44c6-af2a-427055f0026a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:08.978209 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5c6ea1-ca21-4ad6-aa74-c9d260efcbef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:08.978869 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c6ea1-ca21-4ad6-aa74-c9d260efcbef admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:08.979156 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c6ea1-ca21-4ad6-aa74-c9d260efcbef admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:08.979430 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5c6ea1-ca21-4ad6-aa74-c9d260efcbef admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:08.980718 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5c6ea1-ca21-4ad6-aa74-c9d260efcbef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:08.981796 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 887/1797] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:08.982703 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5c6ea1-ca21-4ad6-aa74-c9d260efcbef admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:08.984436 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde015da-f1a9-451f-a70c-7279d9313575 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:08.984891 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde015da-f1a9-451f-a70c-7279d9313575 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:08.985179 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde015da-f1a9-451f-a70c-7279d9313575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:08.996367 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 911/1798] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:08.996367 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5c6ea1-ca21-4ad6-aa74-c9d260efcbef admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:09.005862 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5c6ea1-ca21-4ad6-aa74-c9d260efcbef admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:09.008790 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-bde015da-f1a9-451f-a70c-7279d9313575 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:09.010507 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bde015da-f1a9-451f-a70c-7279d9313575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:09.010690 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd15b9a-d6a8-4385-9e34-c99ca09cc14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:09.012209 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd15b9a-d6a8-4385-9e34-c99ca09cc14f admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:09.012489 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd15b9a-d6a8-4385-9e34-c99ca09cc14f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:09.012766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd15b9a-d6a8-4385-9e34-c99ca09cc14f admin admin] PATH_INFO: `/v3/projects/f84804b2f69d4fa69f9e72a74a2969f1/users/305af93f78a34f4ebfbb2c9503b59852/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:09.022441 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 888/1799] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:09.023814 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde015da-f1a9-451f-a70c-7279d9313575 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:09.031786 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd15b9a-d6a8-4385-9e34-c99ca09cc14f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f84804b2f69d4fa69f9e72a74a2969f1, user_id=305af93f78a34f4ebfbb2c9503b59852, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:09.035461 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd15b9a-d6a8-4385-9e34-c99ca09cc14f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:09.035714 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde015da-f1a9-451f-a70c-7279d9313575 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:09.045333 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bea35a9-4e9b-4ffe-9e06-c1d47fc2fda3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:09.046487 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea35a9-4e9b-4ffe-9e06-c1d47fc2fda3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:09.047705 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea35a9-4e9b-4ffe-9e06-c1d47fc2fda3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:09.048137 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bea35a9-4e9b-4ffe-9e06-c1d47fc2fda3 admin admin] PATH_INFO: `/v3/projects/9093cd0d99bc4514abc8795ef8c3b133/users/9884bb7392b344e288cc0d5307b2c611/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:09.052588 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 912/1800] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:08 2022] PUT /identity/v3/projects/f84804b2f69d4fa69f9e72a74a2969f1/users/305af93f78a34f4ebfbb2c9503b59852/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:09.061664 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd15b9a-d6a8-4385-9e34-c99ca09cc14f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:09.095049 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bea35a9-4e9b-4ffe-9e06-c1d47fc2fda3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9093cd0d99bc4514abc8795ef8c3b133, user_id=9884bb7392b344e288cc0d5307b2c611, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:09.099887 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bea35a9-4e9b-4ffe-9e06-c1d47fc2fda3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:09.197590 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 889/1801] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:09 2022] PUT /identity/v3/projects/9093cd0d99bc4514abc8795ef8c3b133/users/9884bb7392b344e288cc0d5307b2c611/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:09.199215 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33641938-2be3-4107-b67c-c14b714412c0 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:09.199510 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33641938-2be3-4107-b67c-c14b714412c0 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:09.199782 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33641938-2be3-4107-b67c-c14b714412c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:09.203959 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd15b9a-d6a8-4385-9e34-c99ca09cc14f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:09.226685 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-33641938-2be3-4107-b67c-c14b714412c0 None None] MFA Rules not processed for user `305af93f78a34f4ebfbb2c9503b59852`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:09.227928 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-33641938-2be3-4107-b67c-c14b714412c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:09.262688 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b236b8-61ef-4800-bf57-174112604c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:09.263391 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b236b8-61ef-4800-bf57-174112604c95 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:09.263673 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b236b8-61ef-4800-bf57-174112604c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:09.263946 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b236b8-61ef-4800-bf57-174112604c95 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:09.264668 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b236b8-61ef-4800-bf57-174112604c95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:15:09.264970 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b236b8-61ef-4800-bf57-174112604c95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:09.266556 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b236b8-61ef-4800-bf57-174112604c95 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:09.269474 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:09.281474 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 913/1802] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:15:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:15:09.282983 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b236b8-61ef-4800-bf57-174112604c95 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:09.293867 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b236b8-61ef-4800-bf57-174112604c95 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:09.298836 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041d8880-da98-4b88-b20f-a315b96a4117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:09.299517 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041d8880-da98-4b88-b20f-a315b96a4117 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:09.299930 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041d8880-da98-4b88-b20f-a315b96a4117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:09.300198 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041d8880-da98-4b88-b20f-a315b96a4117 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:09.300917 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041d8880-da98-4b88-b20f-a315b96a4117 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:09.302512 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041d8880-da98-4b88-b20f-a315b96a4117 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:09.304396 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 890/1803] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:09.307926 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33641938-2be3-4107-b67c-c14b714412c0 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:09.319113 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 914/1804] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:09.320151 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041d8880-da98-4b88-b20f-a315b96a4117 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:09.333306 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041d8880-da98-4b88-b20f-a315b96a4117 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:09.334640 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33641938-2be3-4107-b67c-c14b714412c0 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:09.338524 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e808acb-83f1-4e75-b78c-1cc979f59202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:09.339914 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e808acb-83f1-4e75-b78c-1cc979f59202 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:09.340468 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e808acb-83f1-4e75-b78c-1cc979f59202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:09.340837 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e808acb-83f1-4e75-b78c-1cc979f59202 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:09.341605 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e808acb-83f1-4e75-b78c-1cc979f59202 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:09.342687 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86cb8b4-785d-4d25-87e2-a6508b044d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:09.343659 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86cb8b4-785d-4d25-87e2-a6508b044d7e admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:09.343944 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86cb8b4-785d-4d25-87e2-a6508b044d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:09.344044 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e808acb-83f1-4e75-b78c-1cc979f59202 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:09.344221 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86cb8b4-785d-4d25-87e2-a6508b044d7e admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:09.344905 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86cb8b4-785d-4d25-87e2-a6508b044d7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:09.347455 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86cb8b4-785d-4d25-87e2-a6508b044d7e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:09.382533 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 891/1805] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:15:09 2022] POST /identity/v3/projects => generated 379 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:09.382533 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86cb8b4-785d-4d25-87e2-a6508b044d7e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:09.382768 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 915/1806] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:15:09 2022] POST /identity/v3/projects => generated 381 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:09.385732 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e808acb-83f1-4e75-b78c-1cc979f59202 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:09.452264 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86cb8b4-785d-4d25-87e2-a6508b044d7e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:09.452875 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e808acb-83f1-4e75-b78c-1cc979f59202 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:09.457128 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c09410-2c8a-4dd6-bfe8-2799599ef877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:09.457549 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60377c5-65a8-404e-9c73-5a5d84f6e8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:09.458011 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c09410-2c8a-4dd6-bfe8-2799599ef877 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:09.458224 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60377c5-65a8-404e-9c73-5a5d84f6e8f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:09.458314 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c09410-2c8a-4dd6-bfe8-2799599ef877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:09.458494 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60377c5-65a8-404e-9c73-5a5d84f6e8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:09.458580 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c09410-2c8a-4dd6-bfe8-2799599ef877 admin admin] PATH_INFO: `/v3/projects/9093cd0d99bc4514abc8795ef8c3b133/users/9884bb7392b344e288cc0d5307b2c611/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:09.458770 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60377c5-65a8-404e-9c73-5a5d84f6e8f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:09.459512 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60377c5-65a8-404e-9c73-5a5d84f6e8f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:09.460536 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c09410-2c8a-4dd6-bfe8-2799599ef877 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9093cd0d99bc4514abc8795ef8c3b133, user_id=9884bb7392b344e288cc0d5307b2c611, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:09.461417 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60377c5-65a8-404e-9c73-5a5d84f6e8f5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:09.463227 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c09410-2c8a-4dd6-bfe8-2799599ef877 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:09.474431 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 892/1807] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:09 2022] PUT /identity/v3/projects/9093cd0d99bc4514abc8795ef8c3b133/users/9884bb7392b344e288cc0d5307b2c611/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:09.475499 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c09410-2c8a-4dd6-bfe8-2799599ef877 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:09.491465 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 916/1808] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:15:09 2022] POST /identity/v3/users => generated 290 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:15:09.492743 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c49002-bdda-44d2-a9ec-dc2ec9326fde None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:09.493035 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c49002-bdda-44d2-a9ec-dc2ec9326fde None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:09.493314 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c49002-bdda-44d2-a9ec-dc2ec9326fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:09.496337 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:09.523079 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-42c49002-bdda-44d2-a9ec-dc2ec9326fde None None] MFA Rules not processed for user `9884bb7392b344e288cc0d5307b2c611`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:09.524288 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-42c49002-bdda-44d2-a9ec-dc2ec9326fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:09.570769 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c09410-2c8a-4dd6-bfe8-2799599ef877 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:09.576790 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987dc0de-2ffc-413b-87e1-03539177dc1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:09.577969 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987dc0de-2ffc-413b-87e1-03539177dc1a admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:09.578385 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987dc0de-2ffc-413b-87e1-03539177dc1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:09.578718 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987dc0de-2ffc-413b-87e1-03539177dc1a admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:09.580165 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987dc0de-2ffc-413b-87e1-03539177dc1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:09.582465 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987dc0de-2ffc-413b-87e1-03539177dc1a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:09.635046 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 893/1809] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:15:09 2022] POST /identity/v3/users => generated 297 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:09.639960 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987dc0de-2ffc-413b-87e1-03539177dc1a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:09.666320 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987dc0de-2ffc-413b-87e1-03539177dc1a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:09.673599 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64da225-2f0b-4bdd-810b-19680657ebd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:09.674549 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64da225-2f0b-4bdd-810b-19680657ebd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:09.674980 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64da225-2f0b-4bdd-810b-19680657ebd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:09.675371 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64da225-2f0b-4bdd-810b-19680657ebd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:09.676877 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64da225-2f0b-4bdd-810b-19680657ebd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:09.691821 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64da225-2f0b-4bdd-810b-19680657ebd0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:09.706751 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 917/1810] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:09 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:09.708719 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c49002-bdda-44d2-a9ec-dc2ec9326fde None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:09.724003 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 894/1811] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:09 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:09.724833 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64da225-2f0b-4bdd-810b-19680657ebd0 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:09.727083 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a64da225-2f0b-4bdd-810b-19680657ebd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:09.732880 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c49002-bdda-44d2-a9ec-dc2ec9326fde None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:09.740368 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaf9793-f051-47f1-acdc-eaca1967aec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:09.741060 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf9793-f051-47f1-acdc-eaca1967aec7 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:09.741343 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf9793-f051-47f1-acdc-eaca1967aec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:09.741808 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf9793-f051-47f1-acdc-eaca1967aec7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:09.742786 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaf9793-f051-47f1-acdc-eaca1967aec7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:09.745715 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaf9793-f051-47f1-acdc-eaca1967aec7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:09.767359 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 918/1812] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:09 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:09.768320 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaf9793-f051-47f1-acdc-eaca1967aec7 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:09.784235 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaf9793-f051-47f1-acdc-eaca1967aec7 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:09.790227 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6842f731-9184-47bc-a8f3-e737ad30e62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:09.790942 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842f731-9184-47bc-a8f3-e737ad30e62c admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:09.791382 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842f731-9184-47bc-a8f3-e737ad30e62c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:09.792334 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842f731-9184-47bc-a8f3-e737ad30e62c admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:09.794042 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6842f731-9184-47bc-a8f3-e737ad30e62c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:09.796218 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6842f731-9184-47bc-a8f3-e737ad30e62c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:09.847264 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 919/1813] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:09 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:09.854203 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6842f731-9184-47bc-a8f3-e737ad30e62c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:09.881988 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64da225-2f0b-4bdd-810b-19680657ebd0 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:09.887442 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd77be4-a04f-48eb-a87f-adddc4e47c73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:09.889233 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd77be4-a04f-48eb-a87f-adddc4e47c73 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:09.889491 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd77be4-a04f-48eb-a87f-adddc4e47c73 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:09.889949 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd77be4-a04f-48eb-a87f-adddc4e47c73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:09.892123 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6842f731-9184-47bc-a8f3-e737ad30e62c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:09.905277 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd77be4-a04f-48eb-a87f-adddc4e47c73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:09.906957 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd77be4-a04f-48eb-a87f-adddc4e47c73 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:09.909278 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c49c6a-4d20-49f2-9174-05ad2d54a1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:09.915237 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c49c6a-4d20-49f2-9174-05ad2d54a1ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:09.915237 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c49c6a-4d20-49f2-9174-05ad2d54a1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:09.915237 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c49c6a-4d20-49f2-9174-05ad2d54a1ca admin admin] PATH_INFO: `/v3/projects/609b83dc143045468ef419ac01900dd9/users/0e265f92773140db93f6bdcef50b7450/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:09.931823 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:09.940091 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c49c6a-4d20-49f2-9174-05ad2d54a1ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=609b83dc143045468ef419ac01900dd9, user_id=0e265f92773140db93f6bdcef50b7450, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:09.947295 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 895/1814] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:09 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:09.950295 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd77be4-a04f-48eb-a87f-adddc4e47c73 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:09.969260 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c49c6a-4d20-49f2-9174-05ad2d54a1ca admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:09.982438 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd77be4-a04f-48eb-a87f-adddc4e47c73 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:09.998575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 920/1815] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:09 2022] PUT /identity/v3/projects/609b83dc143045468ef419ac01900dd9/users/0e265f92773140db93f6bdcef50b7450/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:10.002352 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c49c6a-4d20-49f2-9174-05ad2d54a1ca admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:10.097885 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec234652-8d6c-4902-b2d8-f1b147266729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:10.098793 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec234652-8d6c-4902-b2d8-f1b147266729 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:10.099112 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec234652-8d6c-4902-b2d8-f1b147266729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:10.099389 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec234652-8d6c-4902-b2d8-f1b147266729 admin admin] PATH_INFO: `/v3/projects/f1b3702bc5824688b8232be254ad36be/users/7a5d3623ba0f46cb95710c8f8f1cbcde/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:10.109188 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c49c6a-4d20-49f2-9174-05ad2d54a1ca admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:10.114866 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f146aee-e726-4b27-adb7-7be3e65d9826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:10.115642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f146aee-e726-4b27-adb7-7be3e65d9826 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:10.115926 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f146aee-e726-4b27-adb7-7be3e65d9826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:10.116196 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f146aee-e726-4b27-adb7-7be3e65d9826 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:10.116924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f146aee-e726-4b27-adb7-7be3e65d9826 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:10.118522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f146aee-e726-4b27-adb7-7be3e65d9826 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:10.121748 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec234652-8d6c-4902-b2d8-f1b147266729 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1b3702bc5824688b8232be254ad36be, user_id=7a5d3623ba0f46cb95710c8f8f1cbcde, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:10.127233 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec234652-8d6c-4902-b2d8-f1b147266729 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:10.131937 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 921/1816] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:15:10.136631 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f146aee-e726-4b27-adb7-7be3e65d9826 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:10.152264 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f146aee-e726-4b27-adb7-7be3e65d9826 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:10.152375 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 896/1817] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:09 2022] PUT /identity/v3/projects/f1b3702bc5824688b8232be254ad36be/users/7a5d3623ba0f46cb95710c8f8f1cbcde/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:10.162810 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf8599-4f48-41fa-9394-501e1d783170 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:10.163134 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf8599-4f48-41fa-9394-501e1d783170 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:10.163403 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf8599-4f48-41fa-9394-501e1d783170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:10.207254 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7ebf8599-4f48-41fa-9394-501e1d783170 None None] MFA Rules not processed for user `7a5d3623ba0f46cb95710c8f8f1cbcde`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:10.208530 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7ebf8599-4f48-41fa-9394-501e1d783170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:10.224447 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:10.252318 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9423d5e-203f-4fdb-80e9-d21892fb9f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:10.254128 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9423d5e-203f-4fdb-80e9-d21892fb9f6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:10.254601 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9423d5e-203f-4fdb-80e9-d21892fb9f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:10.255012 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9423d5e-203f-4fdb-80e9-d21892fb9f6a admin admin] PATH_INFO: `/v3/projects/609b83dc143045468ef419ac01900dd9/users/0e265f92773140db93f6bdcef50b7450/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:10.258319 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9423d5e-203f-4fdb-80e9-d21892fb9f6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=609b83dc143045468ef419ac01900dd9, user_id=0e265f92773140db93f6bdcef50b7450, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:10.263357 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9423d5e-203f-4fdb-80e9-d21892fb9f6a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:10.290465 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 922/1818] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:10 2022] PUT /identity/v3/projects/609b83dc143045468ef419ac01900dd9/users/0e265f92773140db93f6bdcef50b7450/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:10.300766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1438d8e-4c20-42c6-99eb-27bc441a0cf8 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:10.300966 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1438d8e-4c20-42c6-99eb-27bc441a0cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:10.301236 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1438d8e-4c20-42c6-99eb-27bc441a0cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:10.307820 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 897/1819] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:15:10.327106 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebf8599-4f48-41fa-9394-501e1d783170 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:10.348983 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebf8599-4f48-41fa-9394-501e1d783170 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:10.367125 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e1438d8e-4c20-42c6-99eb-27bc441a0cf8 None None] MFA Rules not processed for user `0e265f92773140db93f6bdcef50b7450`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:10.370396 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e1438d8e-4c20-42c6-99eb-27bc441a0cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:10.472572 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 923/1820] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:10 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:10.483972 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e83cef-4548-4dec-ae15-03e8433735da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:10.485123 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e83cef-4548-4dec-ae15-03e8433735da admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:10.485576 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e83cef-4548-4dec-ae15-03e8433735da admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:10.486025 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e83cef-4548-4dec-ae15-03e8433735da admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:10.486925 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e83cef-4548-4dec-ae15-03e8433735da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:10.492102 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e83cef-4548-4dec-ae15-03e8433735da admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:10.537619 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 898/1821] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:15:10 2022] POST /identity/v3/projects => generated 379 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:10.541975 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1438d8e-4c20-42c6-99eb-27bc441a0cf8 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:10.562152 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1438d8e-4c20-42c6-99eb-27bc441a0cf8 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:10.573173 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:10.613790 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e83cef-4548-4dec-ae15-03e8433735da admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:10.707738 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541894c8-cd29-4ad5-ab1d-d786fcf21bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:10.708686 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541894c8-cd29-4ad5-ab1d-d786fcf21bec admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:10.709057 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541894c8-cd29-4ad5-ab1d-d786fcf21bec admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:10.709399 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541894c8-cd29-4ad5-ab1d-d786fcf21bec admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:10.710178 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541894c8-cd29-4ad5-ab1d-d786fcf21bec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:10.712436 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541894c8-cd29-4ad5-ab1d-d786fcf21bec admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:10.744142 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 924/1822] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:15:10 2022] POST /identity/v3/users => generated 290 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:15:10.748010 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541894c8-cd29-4ad5-ab1d-d786fcf21bec admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:10.767777 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541894c8-cd29-4ad5-ab1d-d786fcf21bec admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:10.773364 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f8fe92-30c7-4d60-83d8-509a458712bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:10.774042 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f8fe92-30c7-4d60-83d8-509a458712bf admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:10.774468 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f8fe92-30c7-4d60-83d8-509a458712bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:10.775121 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f8fe92-30c7-4d60-83d8-509a458712bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:10.775865 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f8fe92-30c7-4d60-83d8-509a458712bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:10.780030 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f8fe92-30c7-4d60-83d8-509a458712bf admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:10.799347 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 925/1823] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:15:10.801905 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f8fe92-30c7-4d60-83d8-509a458712bf admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:10.823626 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e83cef-4548-4dec-ae15-03e8433735da admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:10.823918 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f8fe92-30c7-4d60-83d8-509a458712bf admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:10.835268 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25f5c99-9972-4f8c-bcdd-5a994b8c628d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:10.836398 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f5c99-9972-4f8c-bcdd-5a994b8c628d service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:10.836398 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f5c99-9972-4f8c-bcdd-5a994b8c628d service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:10.836580 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f5c99-9972-4f8c-bcdd-5a994b8c628d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:10.838882 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7aa4c4-8f45-4a71-86e5-a0e11d46c4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:10.840122 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7aa4c4-8f45-4a71-86e5-a0e11d46c4e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:10.840405 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7aa4c4-8f45-4a71-86e5-a0e11d46c4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:10.840679 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7aa4c4-8f45-4a71-86e5-a0e11d46c4e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:10.841406 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7aa4c4-8f45-4a71-86e5-a0e11d46c4e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:10.843284 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7aa4c4-8f45-4a71-86e5-a0e11d46c4e1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:10.876536 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 926/1824] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:10.878524 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7aa4c4-8f45-4a71-86e5-a0e11d46c4e1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:10.898163 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7aa4c4-8f45-4a71-86e5-a0e11d46c4e1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:10.912405 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45033cde-9088-463b-899e-247e69246946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:10.913949 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45033cde-9088-463b-899e-247e69246946 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:10.914578 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45033cde-9088-463b-899e-247e69246946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:10.915540 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45033cde-9088-463b-899e-247e69246946 admin admin] PATH_INFO: `/v3/projects/c19ff91e06734ff4a115ac01c0f722fb/users/080e88cf0bf648e6a9d36246445b7069/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:10.939022 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:10.948197 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45033cde-9088-463b-899e-247e69246946 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c19ff91e06734ff4a115ac01c0f722fb, user_id=080e88cf0bf648e6a9d36246445b7069, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:10.951782 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45033cde-9088-463b-899e-247e69246946 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:10.959341 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25f5c99-9972-4f8c-bcdd-5a994b8c628d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:10.961252 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25f5c99-9972-4f8c-bcdd-5a994b8c628d service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:10.981377 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 899/1825] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:10 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:15:10.997550 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 927/1826] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:10 2022] PUT /identity/v3/projects/c19ff91e06734ff4a115ac01c0f722fb/users/080e88cf0bf648e6a9d36246445b7069/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:11.010527 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237f12b-c2ea-4d31-b0ea-027ad8584662 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:11.010911 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237f12b-c2ea-4d31-b0ea-027ad8584662 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:11.012851 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4237f12b-c2ea-4d31-b0ea-027ad8584662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:11.081270 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4237f12b-c2ea-4d31-b0ea-027ad8584662 None None] MFA Rules not processed for user `080e88cf0bf648e6a9d36246445b7069`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:11.082743 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4237f12b-c2ea-4d31-b0ea-027ad8584662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:11.280496 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 900/1827] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:11 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:11.284610 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45033cde-9088-463b-899e-247e69246946 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:11.451518 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45033cde-9088-463b-899e-247e69246946 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:11.465196 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69d008f-62c6-4a1b-92e9-44d48f824cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:11.466078 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d008f-62c6-4a1b-92e9-44d48f824cd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:11.466919 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d008f-62c6-4a1b-92e9-44d48f824cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:11.467465 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69d008f-62c6-4a1b-92e9-44d48f824cd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:11.469455 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69d008f-62c6-4a1b-92e9-44d48f824cd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:11.478322 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69d008f-62c6-4a1b-92e9-44d48f824cd0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:11.479584 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237f12b-c2ea-4d31-b0ea-027ad8584662 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:11.495915 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4237f12b-c2ea-4d31-b0ea-027ad8584662 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:11.505389 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d27bce-c09c-465e-a02b-788a4c337cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:11.506370 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d27bce-c09c-465e-a02b-788a4c337cc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:11.506728 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d27bce-c09c-465e-a02b-788a4c337cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:11.507109 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d27bce-c09c-465e-a02b-788a4c337cc2 admin admin] PATH_INFO: `/v3/users/8f512ee1b62147c4927fc0d92a029ebb` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:11.508438 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d27bce-c09c-465e-a02b-788a4c337cc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f512ee1b62147c4927fc0d92a029ebb)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:11.521544 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d27bce-c09c-465e-a02b-788a4c337cc2 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:11.528810 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 928/1828] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:15:11 2022] POST /identity/v3/projects => generated 379 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:11.531232 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69d008f-62c6-4a1b-92e9-44d48f824cd0 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:11.534819 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b69d008f-62c6-4a1b-92e9-44d48f824cd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:11.596042 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:11.618916 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b0d27bce-c09c-465e-a02b-788a4c337cc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f512ee1b62147c4927fc0d92a029ebb'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:11.633977 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69d008f-62c6-4a1b-92e9-44d48f824cd0 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:11.635275 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b0d27bce-c09c-465e-a02b-788a4c337cc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f512ee1b62147c4927fc0d92a029ebb'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:11.640361 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb955c01-d590-4512-b635-7f9130031518 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:11.642345 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb955c01-d590-4512-b635-7f9130031518 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:11.643240 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb955c01-d590-4512-b635-7f9130031518 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:11.643720 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb955c01-d590-4512-b635-7f9130031518 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:11.652410 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b0d27bce-c09c-465e-a02b-788a4c337cc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f512ee1b62147c4927fc0d92a029ebb'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:11.653250 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b0d27bce-c09c-465e-a02b-788a4c337cc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f512ee1b62147c4927fc0d92a029ebb'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:11.653990 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b0d27bce-c09c-465e-a02b-788a4c337cc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f512ee1b62147c4927fc0d92a029ebb'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:11.681565 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 901/1829] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:15:11 2022] DELETE /identity/v3/users/8f512ee1b62147c4927fc0d92a029ebb => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:11.682362 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d27bce-c09c-465e-a02b-788a4c337cc2 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:11.684316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b0d27bce-c09c-465e-a02b-788a4c337cc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:11.760803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb955c01-d590-4512-b635-7f9130031518 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:11.763429 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb955c01-d590-4512-b635-7f9130031518 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:11.765737 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:11.791921 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 929/1830] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:11 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:11.792476 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d27bce-c09c-465e-a02b-788a4c337cc2 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:11.794892 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e0d682-e856-4834-8177-4962f2110a7f None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:11.795499 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e0d682-e856-4834-8177-4962f2110a7f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:11.796538 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e0d682-e856-4834-8177-4962f2110a7f None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:11.800471 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 930/1831] 10.222.0.50 () {54 vars in 921 bytes} [Fri May 27 15:15:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:15:11.803313 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066de827-c588-4538-b2f9-386bbce99cce None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:11.804963 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066de827-c588-4538-b2f9-386bbce99cce None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:11.805344 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066de827-c588-4538-b2f9-386bbce99cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:11.807586 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42782a56-e744-4eb5-a809-4d7fa2f55427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:11.808575 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42782a56-e744-4eb5-a809-4d7fa2f55427 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:11.809048 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42782a56-e744-4eb5-a809-4d7fa2f55427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:11.809335 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42782a56-e744-4eb5-a809-4d7fa2f55427 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:11.810839 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42782a56-e744-4eb5-a809-4d7fa2f55427 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:11.816260 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42782a56-e744-4eb5-a809-4d7fa2f55427 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:11.844296 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 902/1832] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:15:11 2022] POST /identity/v3/users => generated 296 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:11.846487 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975904b6-640d-4595-9e9f-83265acf699e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:11.846708 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975904b6-640d-4595-9e9f-83265acf699e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:11.847025 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975904b6-640d-4595-9e9f-83265acf699e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:11.849980 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-066de827-c588-4538-b2f9-386bbce99cce None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:11.856060 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-066de827-c588-4538-b2f9-386bbce99cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:11.869187 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-975904b6-640d-4595-9e9f-83265acf699e None None] MFA Rules not processed for user `451a9232f95e494683d81b6078a318d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:11.870429 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-975904b6-640d-4595-9e9f-83265acf699e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:11.880414 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 931/1833] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:11.881281 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066de827-c588-4538-b2f9-386bbce99cce None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:11.897226 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066de827-c588-4538-b2f9-386bbce99cce None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:11.913919 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6f5026-e80f-46aa-8866-879724800b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:11.914754 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f5026-e80f-46aa-8866-879724800b46 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:11.915069 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f5026-e80f-46aa-8866-879724800b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:11.915344 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f5026-e80f-46aa-8866-879724800b46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:11.916069 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6f5026-e80f-46aa-8866-879724800b46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:11.917867 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6f5026-e80f-46aa-8866-879724800b46 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:11.935049 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 932/1834] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:11.936499 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6f5026-e80f-46aa-8866-879724800b46 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:11.940577 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bb6f5026-e80f-46aa-8866-879724800b46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:11.987000 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 903/1835] 10.222.0.50 () {58 vars in 1018 bytes} [Fri May 27 15:15:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:11.988495 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975904b6-640d-4595-9e9f-83265acf699e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:12.003191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975904b6-640d-4595-9e9f-83265acf699e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:12.022156 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146a57b6-ea40-411e-b7a0-6fe711555ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:12.038953 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a57b6-ea40-411e-b7a0-6fe711555ef5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:12.039284 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a57b6-ea40-411e-b7a0-6fe711555ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:12.039573 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a57b6-ea40-411e-b7a0-6fe711555ef5 admin admin] PATH_INFO: `/v3/projects/097fe7f4f0e844b98f1254a343a6f24f/users/3b0fda599bb34e378d1a3f7cf66ba5e4/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:12.054740 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6f5026-e80f-46aa-8866-879724800b46 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:12.060052 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f99571-2549-4fc4-8bab-b2d201f9f64b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:12.060969 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f99571-2549-4fc4-8bab-b2d201f9f64b service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:12.061361 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f99571-2549-4fc4-8bab-b2d201f9f64b service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:12.061772 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f99571-2549-4fc4-8bab-b2d201f9f64b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:12.081082 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f99571-2549-4fc4-8bab-b2d201f9f64b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:12.082625 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146a57b6-ea40-411e-b7a0-6fe711555ef5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=097fe7f4f0e844b98f1254a343a6f24f, user_id=3b0fda599bb34e378d1a3f7cf66ba5e4, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:12.083891 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f99571-2549-4fc4-8bab-b2d201f9f64b service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:12.087282 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146a57b6-ea40-411e-b7a0-6fe711555ef5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:12.111661 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 904/1836] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:11 2022] PUT /identity/v3/projects/097fe7f4f0e844b98f1254a343a6f24f/users/3b0fda599bb34e378d1a3f7cf66ba5e4/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:12.113071 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146a57b6-ea40-411e-b7a0-6fe711555ef5 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:12.116885 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-146a57b6-ea40-411e-b7a0-6fe711555ef5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:12.128227 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 933/1837] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:12.128923 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f99571-2549-4fc4-8bab-b2d201f9f64b service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:12.159857 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f99571-2549-4fc4-8bab-b2d201f9f64b service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:12.306061 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa7035-1af1-44a8-9abc-c15c6ba6d178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:12.306740 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa7035-1af1-44a8-9abc-c15c6ba6d178 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:12.307014 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa7035-1af1-44a8-9abc-c15c6ba6d178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:12.307298 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa7035-1af1-44a8-9abc-c15c6ba6d178 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:12.308667 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aa7035-1af1-44a8-9abc-c15c6ba6d178 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:12.312241 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aa7035-1af1-44a8-9abc-c15c6ba6d178 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:12.326816 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146a57b6-ea40-411e-b7a0-6fe711555ef5 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:12.333215 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e440b-fba9-4b27-9da4-32ae1da3378b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '451a9232f95e494683d81b6078a318d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:12.334226 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e440b-fba9-4b27-9da4-32ae1da3378b service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:12.334580 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e440b-fba9-4b27-9da4-32ae1da3378b service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:12.334920 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e440b-fba9-4b27-9da4-32ae1da3378b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:12.340782 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 934/1838] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:12 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:12.345351 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa7035-1af1-44a8-9abc-c15c6ba6d178 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:12.348034 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9e440b-fba9-4b27-9da4-32ae1da3378b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:12.349808 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9e440b-fba9-4b27-9da4-32ae1da3378b service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:12.360297 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa7035-1af1-44a8-9abc-c15c6ba6d178 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:12.369396 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 905/1839] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:12.374777 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f29835-5507-4e5a-89a1-c052fa9bac22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:12.375892 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f29835-5507-4e5a-89a1-c052fa9bac22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:12.376261 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f29835-5507-4e5a-89a1-c052fa9bac22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:12.376602 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f29835-5507-4e5a-89a1-c052fa9bac22 admin admin] PATH_INFO: `/v3/projects/097fe7f4f0e844b98f1254a343a6f24f/users/3b0fda599bb34e378d1a3f7cf66ba5e4/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:12.379422 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f29835-5507-4e5a-89a1-c052fa9bac22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=097fe7f4f0e844b98f1254a343a6f24f, user_id=3b0fda599bb34e378d1a3f7cf66ba5e4, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:12.382959 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f29835-5507-4e5a-89a1-c052fa9bac22 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:12.391240 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:12.402530 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 935/1840] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:12 2022] PUT /identity/v3/projects/097fe7f4f0e844b98f1254a343a6f24f/users/3b0fda599bb34e378d1a3f7cf66ba5e4/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:15:12.413255 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef4ad4-7229-4550-9860-245cc3d0328c None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:12.413539 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef4ad4-7229-4550-9860-245cc3d0328c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:12.413813 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef4ad4-7229-4550-9860-245cc3d0328c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:12.449434 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-deef4ad4-7229-4550-9860-245cc3d0328c None None] MFA Rules not processed for user `3b0fda599bb34e378d1a3f7cf66ba5e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:12.450769 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-deef4ad4-7229-4550-9860-245cc3d0328c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:12.472367 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:12.541363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 906/1841] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:12 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:12.553273 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f29835-5507-4e5a-89a1-c052fa9bac22 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:12.555012 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e4f29835-5507-4e5a-89a1-c052fa9bac22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:12.618046 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f29835-5507-4e5a-89a1-c052fa9bac22 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:12.621794 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d72fb-3abe-4a22-96a7-f39dac2d7a09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:12.622439 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d72fb-3abe-4a22-96a7-f39dac2d7a09 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:12.622724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d72fb-3abe-4a22-96a7-f39dac2d7a09 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:12.623042 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d72fb-3abe-4a22-96a7-f39dac2d7a09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:12.630591 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef4ad4-7229-4550-9860-245cc3d0328c None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:12.632037 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-deef4ad4-7229-4550-9860-245cc3d0328c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:12.633008 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9d72fb-3abe-4a22-96a7-f39dac2d7a09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:12.634599 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9d72fb-3abe-4a22-96a7-f39dac2d7a09 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:12.654959 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 936/1842] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:12 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:12.690278 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d72fb-3abe-4a22-96a7-f39dac2d7a09 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:12.691797 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4b9d72fb-3abe-4a22-96a7-f39dac2d7a09 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:12.703334 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d72fb-3abe-4a22-96a7-f39dac2d7a09 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:12.704355 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef4ad4-7229-4550-9860-245cc3d0328c None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:12.706973 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3280e4-914b-46f7-b90b-b2cb565c237b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:12.707661 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3280e4-914b-46f7-b90b-b2cb565c237b service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:12.707887 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3280e4-914b-46f7-b90b-b2cb565c237b service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:12.708163 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3280e4-914b-46f7-b90b-b2cb565c237b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:12.709316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8604d-769b-46d2-9280-869166352b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:12.709899 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3f3280e4-914b-46f7-b90b-b2cb565c237b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:12.710225 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8604d-769b-46d2-9280-869166352b1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:12.710512 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8604d-769b-46d2-9280-869166352b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:12.710792 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8604d-769b-46d2-9280-869166352b1c admin admin] PATH_INFO: `/v3/projects/aae2b9395d3a48e4be2c252e2ae938db` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:12.712084 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a8604d-769b-46d2-9280-869166352b1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=aae2b9395d3a48e4be2c252e2ae938db)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:12.714025 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a8604d-769b-46d2-9280-869166352b1c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:12.764397 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-51a8604d-769b-46d2-9280-869166352b1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aae2b9395d3a48e4be2c252e2ae938db', 'request_id': 'req-51a8604d-769b-46d2-9280-869166352b1c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:12.769727 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-51a8604d-769b-46d2-9280-869166352b1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aae2b9395d3a48e4be2c252e2ae938db', 'request_id': 'req-51a8604d-769b-46d2-9280-869166352b1c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:12.771466 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3280e4-914b-46f7-b90b-b2cb565c237b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:12.773072 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3280e4-914b-46f7-b90b-b2cb565c237b service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:12.775788 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-51a8604d-769b-46d2-9280-869166352b1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aae2b9395d3a48e4be2c252e2ae938db', 'request_id': 'req-51a8604d-769b-46d2-9280-869166352b1c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:12.783919 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 937/1843] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:12 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:15:12.789657 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-51a8604d-769b-46d2-9280-869166352b1c admin admin] The token cache is being invalidate because project aae2b9395d3a48e4be2c252e2ae938db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:15:12.789953 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-51a8604d-769b-46d2-9280-869166352b1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:12.790598 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-51a8604d-769b-46d2-9280-869166352b1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:12.792204 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 907/1844] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:15:12 2022] DELETE /identity/v3/projects/aae2b9395d3a48e4be2c252e2ae938db => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:12.796375 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3280e4-914b-46f7-b90b-b2cb565c237b service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:12.807276 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3280e4-914b-46f7-b90b-b2cb565c237b service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:12.871537 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6055088-e9d7-490b-a992-7aa91ae44b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:12.873059 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6055088-e9d7-490b-a992-7aa91ae44b23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:12.873059 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6055088-e9d7-490b-a992-7aa91ae44b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:12.873248 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6055088-e9d7-490b-a992-7aa91ae44b23 admin admin] PATH_INFO: `/v3/users/9884bb7392b344e288cc0d5307b2c611` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:12.874544 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6055088-e9d7-490b-a992-7aa91ae44b23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9884bb7392b344e288cc0d5307b2c611)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:12.876780 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6055088-e9d7-490b-a992-7aa91ae44b23 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:12.929575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:12.942384 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c6055088-e9d7-490b-a992-7aa91ae44b23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9884bb7392b344e288cc0d5307b2c611'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:12.952151 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c6055088-e9d7-490b-a992-7aa91ae44b23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9884bb7392b344e288cc0d5307b2c611'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:12.952881 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c6055088-e9d7-490b-a992-7aa91ae44b23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9884bb7392b344e288cc0d5307b2c611'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:12.961848 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c6055088-e9d7-490b-a992-7aa91ae44b23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9884bb7392b344e288cc0d5307b2c611'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:12.963098 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c6055088-e9d7-490b-a992-7aa91ae44b23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9884bb7392b344e288cc0d5307b2c611'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:12.986753 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 938/1845] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:15:12 2022] DELETE /identity/v3/users/9884bb7392b344e288cc0d5307b2c611 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:15:13.346408 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8604d-769b-46d2-9280-869166352b1c admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:13.347410 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-51a8604d-769b-46d2-9280-869166352b1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:13.367849 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8604d-769b-46d2-9280-869166352b1c admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:13.415892 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:13.432393 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a21b64-6044-404d-b78c-e87e6d58c4c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:13.433343 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a21b64-6044-404d-b78c-e87e6d58c4c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:13.433627 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a21b64-6044-404d-b78c-e87e6d58c4c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:13.433909 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a21b64-6044-404d-b78c-e87e6d58c4c9 admin admin] PATH_INFO: `/v3/projects/9093cd0d99bc4514abc8795ef8c3b133` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:13.435120 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a21b64-6044-404d-b78c-e87e6d58c4c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9093cd0d99bc4514abc8795ef8c3b133)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:13.437060 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a21b64-6044-404d-b78c-e87e6d58c4c9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:13.483815 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-18a21b64-6044-404d-b78c-e87e6d58c4c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9093cd0d99bc4514abc8795ef8c3b133', 'request_id': 'req-18a21b64-6044-404d-b78c-e87e6d58c4c9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:13.489270 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-18a21b64-6044-404d-b78c-e87e6d58c4c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9093cd0d99bc4514abc8795ef8c3b133', 'request_id': 'req-18a21b64-6044-404d-b78c-e87e6d58c4c9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:13.495304 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-18a21b64-6044-404d-b78c-e87e6d58c4c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9093cd0d99bc4514abc8795ef8c3b133', 'request_id': 'req-18a21b64-6044-404d-b78c-e87e6d58c4c9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:13.508367 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-18a21b64-6044-404d-b78c-e87e6d58c4c9 admin admin] The token cache is being invalidate because project 9093cd0d99bc4514abc8795ef8c3b133 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:15:13.508660 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-18a21b64-6044-404d-b78c-e87e6d58c4c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:13.509289 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-18a21b64-6044-404d-b78c-e87e6d58c4c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:13.510827 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 908/1846] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:15:13 2022] DELETE /identity/v3/projects/9093cd0d99bc4514abc8795ef8c3b133 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:13.516319 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c0f72-f7fa-4987-9730-23230f5dac3c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:13.516634 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c0f72-f7fa-4987-9730-23230f5dac3c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:13.516933 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c0f72-f7fa-4987-9730-23230f5dac3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:13.535294 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-788c0f72-f7fa-4987-9730-23230f5dac3c None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:13.536610 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-788c0f72-f7fa-4987-9730-23230f5dac3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:13.596400 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 939/1847] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:15:13.604391 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd64bb5-1a82-4615-93fa-1aa98a9995f8 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:13.605219 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd64bb5-1a82-4615-93fa-1aa98a9995f8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:13.605658 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd64bb5-1a82-4615-93fa-1aa98a9995f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:13.625412 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4cd64bb5-1a82-4615-93fa-1aa98a9995f8 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:13.626555 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4cd64bb5-1a82-4615-93fa-1aa98a9995f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:13.634660 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 909/1848] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:13.638925 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788c0f72-f7fa-4987-9730-23230f5dac3c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:13.648966 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788c0f72-f7fa-4987-9730-23230f5dac3c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:13.654629 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c145ac9-c202-420e-93d8-28fdb5131acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:13.655338 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c145ac9-c202-420e-93d8-28fdb5131acc admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:13.655626 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c145ac9-c202-420e-93d8-28fdb5131acc admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:13.656025 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c145ac9-c202-420e-93d8-28fdb5131acc admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:13.656762 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c145ac9-c202-420e-93d8-28fdb5131acc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:15:13.657073 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c145ac9-c202-420e-93d8-28fdb5131acc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:13.658692 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c145ac9-c202-420e-93d8-28fdb5131acc admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:13.672137 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 940/1849] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:15:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:15:13.676235 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd64bb5-1a82-4615-93fa-1aa98a9995f8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:13.685787 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd64bb5-1a82-4615-93fa-1aa98a9995f8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:13.690980 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fbadff-db0f-41d5-a84b-206cad5599d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:13.691651 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fbadff-db0f-41d5-a84b-206cad5599d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:13.691951 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fbadff-db0f-41d5-a84b-206cad5599d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:13.692231 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fbadff-db0f-41d5-a84b-206cad5599d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:13.692935 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fbadff-db0f-41d5-a84b-206cad5599d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:13.694866 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fbadff-db0f-41d5-a84b-206cad5599d4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:13.721816 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 910/1850] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:15:13 2022] POST /identity/v3/projects => generated 401 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:13.726387 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c145ac9-c202-420e-93d8-28fdb5131acc admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:13.736724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c145ac9-c202-420e-93d8-28fdb5131acc admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:13.794637 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e133414e-a355-4763-809e-7e5a38eb3497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:13.795494 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133414e-a355-4763-809e-7e5a38eb3497 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:13.795776 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133414e-a355-4763-809e-7e5a38eb3497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:13.796045 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e133414e-a355-4763-809e-7e5a38eb3497 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:13.796731 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e133414e-a355-4763-809e-7e5a38eb3497 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:13.798671 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e133414e-a355-4763-809e-7e5a38eb3497 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:13.821061 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 941/1851] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:15:13 2022] POST /identity/v3/users => generated 301 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:13.822269 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fbadff-db0f-41d5-a84b-206cad5599d4 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:13.832839 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fbadff-db0f-41d5-a84b-206cad5599d4 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:13.837607 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a35eec0-8e37-4d54-b69c-377aac70af38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:13.838257 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35eec0-8e37-4d54-b69c-377aac70af38 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:13.838561 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35eec0-8e37-4d54-b69c-377aac70af38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:13.838835 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a35eec0-8e37-4d54-b69c-377aac70af38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:13.840362 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a35eec0-8e37-4d54-b69c-377aac70af38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:13.842037 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a35eec0-8e37-4d54-b69c-377aac70af38 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:13.852820 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 911/1852] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:13.857403 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e133414e-a355-4763-809e-7e5a38eb3497 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:13.867005 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e133414e-a355-4763-809e-7e5a38eb3497 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:13.871792 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d869c331-b33b-4086-a346-92c722d33195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:13.872436 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d869c331-b33b-4086-a346-92c722d33195 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:13.872716 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d869c331-b33b-4086-a346-92c722d33195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:13.872995 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d869c331-b33b-4086-a346-92c722d33195 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:13.873704 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d869c331-b33b-4086-a346-92c722d33195 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:13.875331 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d869c331-b33b-4086-a346-92c722d33195 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:13.885262 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 942/1853] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:13 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:13.889181 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a35eec0-8e37-4d54-b69c-377aac70af38 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:13.900411 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a35eec0-8e37-4d54-b69c-377aac70af38 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:13.905874 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8683eb7-d802-4419-bd15-0011de589303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:13.906783 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8683eb7-d802-4419-bd15-0011de589303 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:13.907116 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8683eb7-d802-4419-bd15-0011de589303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:13.907394 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8683eb7-d802-4419-bd15-0011de589303 admin admin] PATH_INFO: `/v3/projects/a2f2efa98746466ba9c45defd8bb8183/users/a2b3a6c41daa40df8c7768e53b1e7835/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:13.923841 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8683eb7-d802-4419-bd15-0011de589303 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2f2efa98746466ba9c45defd8bb8183, user_id=a2b3a6c41daa40df8c7768e53b1e7835, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:13.927233 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8683eb7-d802-4419-bd15-0011de589303 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:13.943517 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 912/1854] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:13 2022] PUT /identity/v3/projects/a2f2efa98746466ba9c45defd8bb8183/users/a2b3a6c41daa40df8c7768e53b1e7835/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:13.954677 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e358a-a2f9-42a0-9f94-943db12aebd4 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:13.955013 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e358a-a2f9-42a0-9f94-943db12aebd4 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:13.955296 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e358a-a2f9-42a0-9f94-943db12aebd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:13.983360 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-314e358a-a2f9-42a0-9f94-943db12aebd4 None None] MFA Rules not processed for user `a2b3a6c41daa40df8c7768e53b1e7835`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:13.984551 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-314e358a-a2f9-42a0-9f94-943db12aebd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:13.996486 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:14.051077 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 943/1855] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:13 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:14.055477 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8683eb7-d802-4419-bd15-0011de589303 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:14.119758 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8683eb7-d802-4419-bd15-0011de589303 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:14.126046 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d17762-eaf7-4ac5-83ca-e6511cdd67df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:14.126718 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d17762-eaf7-4ac5-83ca-e6511cdd67df admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:14.127036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d17762-eaf7-4ac5-83ca-e6511cdd67df admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:14.127316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d17762-eaf7-4ac5-83ca-e6511cdd67df admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:14.128014 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d17762-eaf7-4ac5-83ca-e6511cdd67df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:14.129981 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d17762-eaf7-4ac5-83ca-e6511cdd67df admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:14.160845 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:14.180931 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 913/1856] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:15:14 2022] POST /identity/v3/projects => generated 401 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:14.187054 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314e358a-a2f9-42a0-9f94-943db12aebd4 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:14.212476 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314e358a-a2f9-42a0-9f94-943db12aebd4 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:14.331555 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708ddaf1-88f6-4a10-bcd1-bf9c116e123e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:14.332529 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708ddaf1-88f6-4a10-bcd1-bf9c116e123e admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:14.332825 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708ddaf1-88f6-4a10-bcd1-bf9c116e123e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:14.334939 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708ddaf1-88f6-4a10-bcd1-bf9c116e123e admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:14.334939 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708ddaf1-88f6-4a10-bcd1-bf9c116e123e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:14.337333 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708ddaf1-88f6-4a10-bcd1-bf9c116e123e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:14.373021 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 944/1857] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:15:14 2022] POST /identity/v3/users => generated 307 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:14.377032 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d17762-eaf7-4ac5-83ca-e6511cdd67df admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:14.388378 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d17762-eaf7-4ac5-83ca-e6511cdd67df admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:14.393244 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33d4f8-b70d-4e61-9da2-97351a2141e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:14.393600 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33d4f8-b70d-4e61-9da2-97351a2141e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:14.393961 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33d4f8-b70d-4e61-9da2-97351a2141e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:14.394453 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33d4f8-b70d-4e61-9da2-97351a2141e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:14.395307 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a33d4f8-b70d-4e61-9da2-97351a2141e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:14.398085 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a33d4f8-b70d-4e61-9da2-97351a2141e6 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:14.413638 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708ddaf1-88f6-4a10-bcd1-bf9c116e123e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:14.414762 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 914/1858] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:14.438977 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708ddaf1-88f6-4a10-bcd1-bf9c116e123e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:14.453014 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818874f9-730f-40b1-b338-1e62ecacf213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:14.455215 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818874f9-730f-40b1-b338-1e62ecacf213 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:14.455505 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818874f9-730f-40b1-b338-1e62ecacf213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:14.457089 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818874f9-730f-40b1-b338-1e62ecacf213 admin admin] PATH_INFO: `/v3/projects/6e9a4f4c71054f5391a2fc79b5121292/users/f81db5617339444f80e45610642d9cdf/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:14.500205 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818874f9-730f-40b1-b338-1e62ecacf213 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e9a4f4c71054f5391a2fc79b5121292, user_id=f81db5617339444f80e45610642d9cdf, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:14.503332 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818874f9-730f-40b1-b338-1e62ecacf213 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:14.519664 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 945/1859] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:14 2022] PUT /identity/v3/projects/6e9a4f4c71054f5391a2fc79b5121292/users/f81db5617339444f80e45610642d9cdf/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:14.524071 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33d4f8-b70d-4e61-9da2-97351a2141e6 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:14.536465 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33d4f8-b70d-4e61-9da2-97351a2141e6 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:14.603769 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ea946-4db2-41c9-85e7-d6ed6b3f5b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:14.604458 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ea946-4db2-41c9-85e7-d6ed6b3f5b5f admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:14.605322 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ea946-4db2-41c9-85e7-d6ed6b3f5b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:14.605887 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ea946-4db2-41c9-85e7-d6ed6b3f5b5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:14.606811 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602ea946-4db2-41c9-85e7-d6ed6b3f5b5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:14.608473 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602ea946-4db2-41c9-85e7-d6ed6b3f5b5f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:14.619827 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 915/1860] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:14.624280 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818874f9-730f-40b1-b338-1e62ecacf213 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:14.635373 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818874f9-730f-40b1-b338-1e62ecacf213 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:14.641109 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0637f7e5-e908-4163-9e98-868e292a8ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:14.641996 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0637f7e5-e908-4163-9e98-868e292a8ad1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:14.642261 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0637f7e5-e908-4163-9e98-868e292a8ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:14.643120 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0637f7e5-e908-4163-9e98-868e292a8ad1 admin admin] PATH_INFO: `/v3/projects/6e9a4f4c71054f5391a2fc79b5121292/users/f81db5617339444f80e45610642d9cdf/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:14.648746 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0637f7e5-e908-4163-9e98-868e292a8ad1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e9a4f4c71054f5391a2fc79b5121292, user_id=f81db5617339444f80e45610642d9cdf, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:14.652359 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0637f7e5-e908-4163-9e98-868e292a8ad1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:14.666868 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 946/1861] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:14 2022] PUT /identity/v3/projects/6e9a4f4c71054f5391a2fc79b5121292/users/f81db5617339444f80e45610642d9cdf/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:15:14.678192 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593562d-1505-4bbb-9184-cd737165189a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:14.678683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593562d-1505-4bbb-9184-cd737165189a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:14.679346 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c593562d-1505-4bbb-9184-cd737165189a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:14.708719 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c593562d-1505-4bbb-9184-cd737165189a None None] MFA Rules not processed for user `f81db5617339444f80e45610642d9cdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:14.710254 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c593562d-1505-4bbb-9184-cd737165189a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:14.826773 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 916/1862] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:14 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:14.842316 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0637f7e5-e908-4163-9e98-868e292a8ad1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:14.865044 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0637f7e5-e908-4163-9e98-868e292a8ad1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:15.012816 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0637f7e5-e908-4163-9e98-868e292a8ad1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:15.024917 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75093a5e-fd89-4dc7-af73-af7480c62f24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:15.029077 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75093a5e-fd89-4dc7-af73-af7480c62f24 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:15.029374 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75093a5e-fd89-4dc7-af73-af7480c62f24 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:15.029664 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75093a5e-fd89-4dc7-af73-af7480c62f24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:15.062881 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75093a5e-fd89-4dc7-af73-af7480c62f24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:15.064369 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75093a5e-fd89-4dc7-af73-af7480c62f24 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:15.076785 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 947/1863] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:14 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:15.234820 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c593562d-1505-4bbb-9184-cd737165189a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:15.236953 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c593562d-1505-4bbb-9184-cd737165189a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:15.272520 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c593562d-1505-4bbb-9184-cd737165189a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:15.289399 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa590193-db77-461b-b360-4b594b49fed9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:15.290328 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa590193-db77-461b-b360-4b594b49fed9 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:15.290803 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa590193-db77-461b-b360-4b594b49fed9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:15.294255 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa590193-db77-461b-b360-4b594b49fed9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:15.452093 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa590193-db77-461b-b360-4b594b49fed9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:15.454373 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa590193-db77-461b-b360-4b594b49fed9 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:15.492011 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 917/1864] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:15 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:16.893813 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c636d92-0ccf-482b-92ca-9291b6ce5b65 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:16.894614 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c636d92-0ccf-482b-92ca-9291b6ce5b65 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:16.894614 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c636d92-0ccf-482b-92ca-9291b6ce5b65 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:16.896431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 948/1865] 10.222.0.50 () {54 vars in 921 bytes} [Fri May 27 15:15:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:15:16.907092 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f312e-70d5-403e-9ab8-f0aaf5311927 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:16.907628 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f312e-70d5-403e-9ab8-f0aaf5311927 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:16.908240 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f312e-70d5-403e-9ab8-f0aaf5311927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:16.939796 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:16.990536 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-444f312e-70d5-403e-9ab8-f0aaf5311927 None None] MFA Rules not processed for user `451a9232f95e494683d81b6078a318d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:16.994302 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-444f312e-70d5-403e-9ab8-f0aaf5311927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:17.142379 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 918/1866] 10.222.0.50 () {58 vars in 1018 bytes} [Fri May 27 15:15:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:17.371390 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c636d92-0ccf-482b-92ca-9291b6ce5b65 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:17.373841 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9c636d92-0ccf-482b-92ca-9291b6ce5b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:17.412232 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c636d92-0ccf-482b-92ca-9291b6ce5b65 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:17.433938 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c29f18-2ac6-41a6-b703-b674bb945f6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '451a9232f95e494683d81b6078a318d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:17.435137 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c29f18-2ac6-41a6-b703-b674bb945f6d service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:17.435573 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c29f18-2ac6-41a6-b703-b674bb945f6d service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:17.436058 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c29f18-2ac6-41a6-b703-b674bb945f6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:17.448084 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:17.455046 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c29f18-2ac6-41a6-b703-b674bb945f6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:17.459671 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c29f18-2ac6-41a6-b703-b674bb945f6d service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:17.474426 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 949/1867] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:17.614034 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f312e-70d5-403e-9ab8-f0aaf5311927 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:17.615469 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-444f312e-70d5-403e-9ab8-f0aaf5311927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:17.695474 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f312e-70d5-403e-9ab8-f0aaf5311927 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:17.700946 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f01e1f-e7d3-4fe2-823a-eaa8779f2e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:17.701835 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f01e1f-e7d3-4fe2-823a-eaa8779f2e20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:17.702215 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f01e1f-e7d3-4fe2-823a-eaa8779f2e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:17.702523 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f01e1f-e7d3-4fe2-823a-eaa8779f2e20 admin admin] PATH_INFO: `/v3/users/305af93f78a34f4ebfbb2c9503b59852` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:17.703834 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f01e1f-e7d3-4fe2-823a-eaa8779f2e20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=305af93f78a34f4ebfbb2c9503b59852)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:17.705873 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f01e1f-e7d3-4fe2-823a-eaa8779f2e20 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:17.755268 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-97f01e1f-e7d3-4fe2-823a-eaa8779f2e20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '305af93f78a34f4ebfbb2c9503b59852'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:17.762807 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-97f01e1f-e7d3-4fe2-823a-eaa8779f2e20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '305af93f78a34f4ebfbb2c9503b59852'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:17.770192 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-97f01e1f-e7d3-4fe2-823a-eaa8779f2e20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '305af93f78a34f4ebfbb2c9503b59852'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:17.770843 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-97f01e1f-e7d3-4fe2-823a-eaa8779f2e20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '305af93f78a34f4ebfbb2c9503b59852'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:17.771490 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-97f01e1f-e7d3-4fe2-823a-eaa8779f2e20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '305af93f78a34f4ebfbb2c9503b59852'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:17.790712 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 919/1868] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:15:17 2022] DELETE /identity/v3/users/305af93f78a34f4ebfbb2c9503b59852 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:17.796083 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94014c25-20e6-4d9b-8528-aac90ad39b76 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:17.796378 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94014c25-20e6-4d9b-8528-aac90ad39b76 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:17.796661 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94014c25-20e6-4d9b-8528-aac90ad39b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:17.816442 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-94014c25-20e6-4d9b-8528-aac90ad39b76 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:17.817644 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-94014c25-20e6-4d9b-8528-aac90ad39b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:17.892347 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 950/1869] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:15:17.906036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f01e1f-e7d3-4fe2-823a-eaa8779f2e20 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:17.907469 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-97f01e1f-e7d3-4fe2-823a-eaa8779f2e20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:17.932056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:17.987604 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f01e1f-e7d3-4fe2-823a-eaa8779f2e20 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:17.991526 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d243e55c-23ea-4b75-934b-6aef8e70e182 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:17.992191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243e55c-23ea-4b75-934b-6aef8e70e182 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:17.992503 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243e55c-23ea-4b75-934b-6aef8e70e182 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:17.992750 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243e55c-23ea-4b75-934b-6aef8e70e182 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:18.003462 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d243e55c-23ea-4b75-934b-6aef8e70e182 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:18.004903 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d243e55c-23ea-4b75-934b-6aef8e70e182 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:18.016554 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 920/1870] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:18.221887 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94014c25-20e6-4d9b-8528-aac90ad39b76 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:18.223433 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-94014c25-20e6-4d9b-8528-aac90ad39b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:18.247473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94014c25-20e6-4d9b-8528-aac90ad39b76 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:18.255526 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae0f576-2b1a-4adb-b34b-b7defa9b83f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:18.257723 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae0f576-2b1a-4adb-b34b-b7defa9b83f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:18.258214 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae0f576-2b1a-4adb-b34b-b7defa9b83f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:18.258496 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae0f576-2b1a-4adb-b34b-b7defa9b83f1 admin admin] PATH_INFO: `/v3/users/7a5d3623ba0f46cb95710c8f8f1cbcde` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:18.259956 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae0f576-2b1a-4adb-b34b-b7defa9b83f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a5d3623ba0f46cb95710c8f8f1cbcde)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:18.262322 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae0f576-2b1a-4adb-b34b-b7defa9b83f1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:18.352956 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0ae0f576-2b1a-4adb-b34b-b7defa9b83f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a5d3623ba0f46cb95710c8f8f1cbcde'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:18.386939 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0ae0f576-2b1a-4adb-b34b-b7defa9b83f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a5d3623ba0f46cb95710c8f8f1cbcde'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:18.390685 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0ae0f576-2b1a-4adb-b34b-b7defa9b83f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a5d3623ba0f46cb95710c8f8f1cbcde'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:18.411516 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0ae0f576-2b1a-4adb-b34b-b7defa9b83f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a5d3623ba0f46cb95710c8f8f1cbcde'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:18.412488 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0ae0f576-2b1a-4adb-b34b-b7defa9b83f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a5d3623ba0f46cb95710c8f8f1cbcde'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:18.469575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 951/1871] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:15:18 2022] DELETE /identity/v3/users/7a5d3623ba0f46cb95710c8f8f1cbcde => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:18.474167 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb2267-515e-49bf-a0a2-cb1c0d70ec32 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:18.474480 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb2267-515e-49bf-a0a2-cb1c0d70ec32 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:18.474723 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceb2267-515e-49bf-a0a2-cb1c0d70ec32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:18.516892 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4ceb2267-515e-49bf-a0a2-cb1c0d70ec32 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:18.518157 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4ceb2267-515e-49bf-a0a2-cb1c0d70ec32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:18.655500 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 921/1872] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:18.691477 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae0f576-2b1a-4adb-b34b-b7defa9b83f1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:18.693238 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0ae0f576-2b1a-4adb-b34b-b7defa9b83f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:18.844331 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae0f576-2b1a-4adb-b34b-b7defa9b83f1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:18.853101 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd307c8-61aa-47d3-ac61-41c19c970d59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:18.854126 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd307c8-61aa-47d3-ac61-41c19c970d59 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:18.854585 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd307c8-61aa-47d3-ac61-41c19c970d59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:18.855073 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd307c8-61aa-47d3-ac61-41c19c970d59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:18.874706 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd307c8-61aa-47d3-ac61-41c19c970d59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:18.876253 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd307c8-61aa-47d3-ac61-41c19c970d59 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:18.903140 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 952/1873] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:15:18.931574 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb2267-515e-49bf-a0a2-cb1c0d70ec32 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:18.938082 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4ceb2267-515e-49bf-a0a2-cb1c0d70ec32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:18.976621 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceb2267-515e-49bf-a0a2-cb1c0d70ec32 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:18.993222 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c78423e-7eb9-4c68-92bc-39f19a5c60d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:18.995127 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78423e-7eb9-4c68-92bc-39f19a5c60d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:18.999453 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78423e-7eb9-4c68-92bc-39f19a5c60d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:18.999868 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78423e-7eb9-4c68-92bc-39f19a5c60d0 admin admin] PATH_INFO: `/v3/projects/f84804b2f69d4fa69f9e72a74a2969f1` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:19.001456 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c78423e-7eb9-4c68-92bc-39f19a5c60d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f84804b2f69d4fa69f9e72a74a2969f1)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:19.004540 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c78423e-7eb9-4c68-92bc-39f19a5c60d0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:19.085066 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4c78423e-7eb9-4c68-92bc-39f19a5c60d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f84804b2f69d4fa69f9e72a74a2969f1', 'request_id': 'req-4c78423e-7eb9-4c68-92bc-39f19a5c60d0'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:19.092704 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4c78423e-7eb9-4c68-92bc-39f19a5c60d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f84804b2f69d4fa69f9e72a74a2969f1', 'request_id': 'req-4c78423e-7eb9-4c68-92bc-39f19a5c60d0'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:19.103684 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4c78423e-7eb9-4c68-92bc-39f19a5c60d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f84804b2f69d4fa69f9e72a74a2969f1', 'request_id': 'req-4c78423e-7eb9-4c68-92bc-39f19a5c60d0'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:19.120976 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4c78423e-7eb9-4c68-92bc-39f19a5c60d0 admin admin] The token cache is being invalidate because project f84804b2f69d4fa69f9e72a74a2969f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:15:19.121398 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4c78423e-7eb9-4c68-92bc-39f19a5c60d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:19.122458 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4c78423e-7eb9-4c68-92bc-39f19a5c60d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:19.126108 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 922/1874] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:15:18 2022] DELETE /identity/v3/projects/f84804b2f69d4fa69f9e72a74a2969f1 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:19.129036 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd307c8-61aa-47d3-ac61-41c19c970d59 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:19.142959 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd307c8-61aa-47d3-ac61-41c19c970d59 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:19.160777 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:19.244264 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d981e938-a790-4f0e-867c-c9d525a93383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:19.245334 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981e938-a790-4f0e-867c-c9d525a93383 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:19.245615 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981e938-a790-4f0e-867c-c9d525a93383 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:19.246269 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d981e938-a790-4f0e-867c-c9d525a93383 admin admin] PATH_INFO: `/v3/users/0e265f92773140db93f6bdcef50b7450` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:19.247866 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d981e938-a790-4f0e-867c-c9d525a93383 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e265f92773140db93f6bdcef50b7450)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:19.252408 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d981e938-a790-4f0e-867c-c9d525a93383 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:19.331060 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d981e938-a790-4f0e-867c-c9d525a93383 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e265f92773140db93f6bdcef50b7450'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:19.340826 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d981e938-a790-4f0e-867c-c9d525a93383 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e265f92773140db93f6bdcef50b7450'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:19.342349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d981e938-a790-4f0e-867c-c9d525a93383 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e265f92773140db93f6bdcef50b7450'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:19.354058 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d981e938-a790-4f0e-867c-c9d525a93383 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e265f92773140db93f6bdcef50b7450'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:19.354742 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d981e938-a790-4f0e-867c-c9d525a93383 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e265f92773140db93f6bdcef50b7450'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:19.378649 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 953/1875] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:15:19 2022] DELETE /identity/v3/users/0e265f92773140db93f6bdcef50b7450 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:19.714244 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c78423e-7eb9-4c68-92bc-39f19a5c60d0 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:19.716093 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4c78423e-7eb9-4c68-92bc-39f19a5c60d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:19.741093 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c78423e-7eb9-4c68-92bc-39f19a5c60d0 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:19.798637 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d981e938-a790-4f0e-867c-c9d525a93383 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:19.799311 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:19.800093 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d981e938-a790-4f0e-867c-c9d525a93383 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:19.820004 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a6912b-b89b-47be-8a14-b2d3cfda6e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:19.821165 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6912b-b89b-47be-8a14-b2d3cfda6e29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:19.821449 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6912b-b89b-47be-8a14-b2d3cfda6e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:19.821725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6912b-b89b-47be-8a14-b2d3cfda6e29 admin admin] PATH_INFO: `/v3/projects/f1b3702bc5824688b8232be254ad36be` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:19.822940 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a6912b-b89b-47be-8a14-b2d3cfda6e29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1b3702bc5824688b8232be254ad36be)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:19.825420 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a6912b-b89b-47be-8a14-b2d3cfda6e29 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:19.876652 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d981e938-a790-4f0e-867c-c9d525a93383 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:19.877937 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-63a6912b-b89b-47be-8a14-b2d3cfda6e29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1b3702bc5824688b8232be254ad36be', 'request_id': 'req-63a6912b-b89b-47be-8a14-b2d3cfda6e29'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:19.883612 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-63a6912b-b89b-47be-8a14-b2d3cfda6e29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1b3702bc5824688b8232be254ad36be', 'request_id': 'req-63a6912b-b89b-47be-8a14-b2d3cfda6e29'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:19.890260 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-63a6912b-b89b-47be-8a14-b2d3cfda6e29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1b3702bc5824688b8232be254ad36be', 'request_id': 'req-63a6912b-b89b-47be-8a14-b2d3cfda6e29'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:19.904673 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-63a6912b-b89b-47be-8a14-b2d3cfda6e29 admin admin] The token cache is being invalidate because project f1b3702bc5824688b8232be254ad36be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:15:19.904965 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-63a6912b-b89b-47be-8a14-b2d3cfda6e29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:19.905599 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-63a6912b-b89b-47be-8a14-b2d3cfda6e29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:19.907466 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 923/1876] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:15:19 2022] DELETE /identity/v3/projects/f1b3702bc5824688b8232be254ad36be => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:19.911119 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a6912b-b89b-47be-8a14-b2d3cfda6e29 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:19.912304 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:19.912539 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-63a6912b-b89b-47be-8a14-b2d3cfda6e29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:19.941707 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70c69b3-b9f5-40b7-bc47-3709e2b801c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:19.942578 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c69b3-b9f5-40b7-bc47-3709e2b801c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:19.942885 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c69b3-b9f5-40b7-bc47-3709e2b801c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:19.943168 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70c69b3-b9f5-40b7-bc47-3709e2b801c7 admin admin] PATH_INFO: `/v3/projects/609b83dc143045468ef419ac01900dd9` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:19.944347 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70c69b3-b9f5-40b7-bc47-3709e2b801c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=609b83dc143045468ef419ac01900dd9)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:19.946349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70c69b3-b9f5-40b7-bc47-3709e2b801c7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:19.988567 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a6912b-b89b-47be-8a14-b2d3cfda6e29 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:19.993680 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b162076-e22b-4f87-9d7c-1a521fb622e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:19.994834 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b162076-e22b-4f87-9d7c-1a521fb622e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:19.995503 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b162076-e22b-4f87-9d7c-1a521fb622e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:19.996680 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b162076-e22b-4f87-9d7c-1a521fb622e1 admin admin] PATH_INFO: `/v3/users/080e88cf0bf648e6a9d36246445b7069` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:19.998025 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b162076-e22b-4f87-9d7c-1a521fb622e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=080e88cf0bf648e6a9d36246445b7069)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:20.000232 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b162076-e22b-4f87-9d7c-1a521fb622e1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:20.008416 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a70c69b3-b9f5-40b7-bc47-3709e2b801c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '609b83dc143045468ef419ac01900dd9', 'request_id': 'req-a70c69b3-b9f5-40b7-bc47-3709e2b801c7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:20.016457 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a70c69b3-b9f5-40b7-bc47-3709e2b801c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '609b83dc143045468ef419ac01900dd9', 'request_id': 'req-a70c69b3-b9f5-40b7-bc47-3709e2b801c7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:20.023693 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a70c69b3-b9f5-40b7-bc47-3709e2b801c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '609b83dc143045468ef419ac01900dd9', 'request_id': 'req-a70c69b3-b9f5-40b7-bc47-3709e2b801c7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:20.040272 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a70c69b3-b9f5-40b7-bc47-3709e2b801c7 admin admin] The token cache is being invalidate because project 609b83dc143045468ef419ac01900dd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:15:20.040667 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a70c69b3-b9f5-40b7-bc47-3709e2b801c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:20.041465 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a70c69b3-b9f5-40b7-bc47-3709e2b801c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:20.046758 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 954/1877] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:15:19 2022] DELETE /identity/v3/projects/609b83dc143045468ef419ac01900dd9 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:15:20.049216 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ead795-09f9-4493-a069-29681346d46a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:20.049678 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ead795-09f9-4493-a069-29681346d46a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:20.050038 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ead795-09f9-4493-a069-29681346d46a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:20.063074 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8b162076-e22b-4f87-9d7c-1a521fb622e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '080e88cf0bf648e6a9d36246445b7069'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:20.071688 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8b162076-e22b-4f87-9d7c-1a521fb622e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '080e88cf0bf648e6a9d36246445b7069'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:20.072677 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f0ead795-09f9-4493-a069-29681346d46a None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:20.074956 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f0ead795-09f9-4493-a069-29681346d46a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:20.080794 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8b162076-e22b-4f87-9d7c-1a521fb622e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '080e88cf0bf648e6a9d36246445b7069'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:20.082803 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8b162076-e22b-4f87-9d7c-1a521fb622e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '080e88cf0bf648e6a9d36246445b7069'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:20.083717 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8b162076-e22b-4f87-9d7c-1a521fb622e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '080e88cf0bf648e6a9d36246445b7069'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:20.108426 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:20.121064 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 924/1878] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:15:19 2022] DELETE /identity/v3/users/080e88cf0bf648e6a9d36246445b7069 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:15:20.159225 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 955/1879] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:20.168084 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b42dd-651d-48bb-80b2-5a1b99771607 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:20.168526 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b42dd-651d-48bb-80b2-5a1b99771607 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:20.168908 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b42dd-651d-48bb-80b2-5a1b99771607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:20.192341 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-af2b42dd-651d-48bb-80b2-5a1b99771607 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:20.193670 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-af2b42dd-651d-48bb-80b2-5a1b99771607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:20.298075 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ead795-09f9-4493-a069-29681346d46a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:20.306783 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 925/1880] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:20.314139 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ead795-09f9-4493-a069-29681346d46a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:20.319264 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4d4ffe-29db-4a68-a004-b45c6bfbb2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:20.319940 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d4ffe-29db-4a68-a004-b45c6bfbb2a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:20.320397 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d4ffe-29db-4a68-a004-b45c6bfbb2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:20.321015 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d4ffe-29db-4a68-a004-b45c6bfbb2a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:20.322302 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4d4ffe-29db-4a68-a004-b45c6bfbb2a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:15:20.323139 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4d4ffe-29db-4a68-a004-b45c6bfbb2a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:20.326617 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4d4ffe-29db-4a68-a004-b45c6bfbb2a4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:20.349981 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b42dd-651d-48bb-80b2-5a1b99771607 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:20.358566 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 956/1881] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:15:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:15:20.367015 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b42dd-651d-48bb-80b2-5a1b99771607 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:20.383882 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1661f8-1409-494f-ac49-9391c9d194e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:20.384646 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1661f8-1409-494f-ac49-9391c9d194e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:20.386428 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1661f8-1409-494f-ac49-9391c9d194e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:20.386428 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1661f8-1409-494f-ac49-9391c9d194e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:20.386717 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1661f8-1409-494f-ac49-9391c9d194e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:20.399406 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1661f8-1409-494f-ac49-9391c9d194e5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:20.459131 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 926/1882] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:15:20 2022] POST /identity/v3/projects => generated 389 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:20.459372 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4d4ffe-29db-4a68-a004-b45c6bfbb2a4 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:20.471814 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4d4ffe-29db-4a68-a004-b45c6bfbb2a4 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:20.616191 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:20.642287 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea95bc4-65a9-40d1-93a6-ef2811ee9b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:20.643200 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea95bc4-65a9-40d1-93a6-ef2811ee9b5d admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:20.643764 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea95bc4-65a9-40d1-93a6-ef2811ee9b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:20.644192 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea95bc4-65a9-40d1-93a6-ef2811ee9b5d admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:20.645044 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea95bc4-65a9-40d1-93a6-ef2811ee9b5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:20.647533 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea95bc4-65a9-40d1-93a6-ef2811ee9b5d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:20.687387 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 957/1883] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:15:20 2022] POST /identity/v3/users => generated 295 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:20.691388 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1661f8-1409-494f-ac49-9391c9d194e5 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:20.708258 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea95bc4-65a9-40d1-93a6-ef2811ee9b5d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:20.710748 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5ea95bc4-65a9-40d1-93a6-ef2811ee9b5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:20.713439 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1661f8-1409-494f-ac49-9391c9d194e5 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:20.721649 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66a0fcb-82a3-40af-8527-63a4a1ce73ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:20.722496 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a0fcb-82a3-40af-8527-63a4a1ce73ea admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:20.723423 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a0fcb-82a3-40af-8527-63a4a1ce73ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:20.723805 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a0fcb-82a3-40af-8527-63a4a1ce73ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:20.724861 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66a0fcb-82a3-40af-8527-63a4a1ce73ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:20.727056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66a0fcb-82a3-40af-8527-63a4a1ce73ea admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:20.749458 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea95bc4-65a9-40d1-93a6-ef2811ee9b5d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:20.749948 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 927/1884] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:20.750906 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66a0fcb-82a3-40af-8527-63a4a1ce73ea admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:20.753477 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d69bf87-2842-46ab-9967-54674ef2ac2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:20.754693 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69bf87-2842-46ab-9967-54674ef2ac2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:20.755109 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69bf87-2842-46ab-9967-54674ef2ac2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:20.755394 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69bf87-2842-46ab-9967-54674ef2ac2e admin admin] PATH_INFO: `/v3/users/a2b3a6c41daa40df8c7768e53b1e7835` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:20.756629 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d69bf87-2842-46ab-9967-54674ef2ac2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2b3a6c41daa40df8c7768e53b1e7835)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:20.758561 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d69bf87-2842-46ab-9967-54674ef2ac2e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:20.766744 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66a0fcb-82a3-40af-8527-63a4a1ce73ea admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:20.778804 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d495cc0-d8a5-4c6e-8e52-fff1b371e335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:20.779773 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d495cc0-d8a5-4c6e-8e52-fff1b371e335 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:20.780778 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d495cc0-d8a5-4c6e-8e52-fff1b371e335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:20.783226 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d495cc0-d8a5-4c6e-8e52-fff1b371e335 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:20.784524 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d495cc0-d8a5-4c6e-8e52-fff1b371e335 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:20.786268 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d495cc0-d8a5-4c6e-8e52-fff1b371e335 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:20.816512 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 928/1885] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:20.822043 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d495cc0-d8a5-4c6e-8e52-fff1b371e335 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:20.850015 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d495cc0-d8a5-4c6e-8e52-fff1b371e335 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:20.863271 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9723ba9f-bef6-45a0-8e02-a78f50694b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:20.866637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723ba9f-bef6-45a0-8e02-a78f50694b32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:20.867112 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723ba9f-bef6-45a0-8e02-a78f50694b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:20.867402 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723ba9f-bef6-45a0-8e02-a78f50694b32 admin admin] PATH_INFO: `/v3/projects/99da31ee7fe74f2bbd6789f4e2a3b696/users/d1580f3882134b7dbcbfbf736771dfd6/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:20.872130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2d69bf87-2842-46ab-9967-54674ef2ac2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2b3a6c41daa40df8c7768e53b1e7835'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:20.880527 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2d69bf87-2842-46ab-9967-54674ef2ac2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2b3a6c41daa40df8c7768e53b1e7835'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:20.881423 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2d69bf87-2842-46ab-9967-54674ef2ac2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2b3a6c41daa40df8c7768e53b1e7835'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:20.890671 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2d69bf87-2842-46ab-9967-54674ef2ac2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2b3a6c41daa40df8c7768e53b1e7835'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:20.891721 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2d69bf87-2842-46ab-9967-54674ef2ac2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2b3a6c41daa40df8c7768e53b1e7835'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:20.900245 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9723ba9f-bef6-45a0-8e02-a78f50694b32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99da31ee7fe74f2bbd6789f4e2a3b696, user_id=d1580f3882134b7dbcbfbf736771dfd6, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:20.905454 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9723ba9f-bef6-45a0-8e02-a78f50694b32 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:20.931073 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 958/1887] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:15:20 2022] DELETE /identity/v3/users/a2b3a6c41daa40df8c7768e53b1e7835 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:15:20.932000 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d69bf87-2842-46ab-9967-54674ef2ac2e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:20.932852 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 929/1887] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:20 2022] PUT /identity/v3/projects/99da31ee7fe74f2bbd6789f4e2a3b696/users/d1580f3882134b7dbcbfbf736771dfd6/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:20.934236 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2d69bf87-2842-46ab-9967-54674ef2ac2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:20.934442 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da154b0-3931-408f-bc1e-21598aa57f2a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:20.934741 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da154b0-3931-408f-bc1e-21598aa57f2a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:20.937762 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da154b0-3931-408f-bc1e-21598aa57f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:20.972174 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8da154b0-3931-408f-bc1e-21598aa57f2a None None] MFA Rules not processed for user `d1580f3882134b7dbcbfbf736771dfd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:20.973762 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8da154b0-3931-408f-bc1e-21598aa57f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:21.031277 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d69bf87-2842-46ab-9967-54674ef2ac2e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:21.036416 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a062070-d51a-4c86-93e1-db261a697e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:21.037385 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a062070-d51a-4c86-93e1-db261a697e3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:21.037739 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a062070-d51a-4c86-93e1-db261a697e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:21.038085 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a062070-d51a-4c86-93e1-db261a697e3e admin admin] PATH_INFO: `/v3/projects/c19ff91e06734ff4a115ac01c0f722fb` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:21.039619 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a062070-d51a-4c86-93e1-db261a697e3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c19ff91e06734ff4a115ac01c0f722fb)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:21.042328 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a062070-d51a-4c86-93e1-db261a697e3e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:21.060716 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 930/1888] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:21.063501 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181beec5-3571-4c33-a0e8-2a94a03bf9ba None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:21.063951 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181beec5-3571-4c33-a0e8-2a94a03bf9ba None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:21.064623 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181beec5-3571-4c33-a0e8-2a94a03bf9ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:21.084888 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-181beec5-3571-4c33-a0e8-2a94a03bf9ba None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:21.086390 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-181beec5-3571-4c33-a0e8-2a94a03bf9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:21.097081 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 931/1889] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:21.098120 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181beec5-3571-4c33-a0e8-2a94a03bf9ba None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:21.099601 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-181beec5-3571-4c33-a0e8-2a94a03bf9ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:21.113711 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181beec5-3571-4c33-a0e8-2a94a03bf9ba None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:21.118987 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:21.140903 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7a062070-d51a-4c86-93e1-db261a697e3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c19ff91e06734ff4a115ac01c0f722fb', 'request_id': 'req-7a062070-d51a-4c86-93e1-db261a697e3e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:21.162328 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7a062070-d51a-4c86-93e1-db261a697e3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c19ff91e06734ff4a115ac01c0f722fb', 'request_id': 'req-7a062070-d51a-4c86-93e1-db261a697e3e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:21.170407 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=79,keystone:INSERT=5,keystone:DELETE=16 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:21.177876 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7a062070-d51a-4c86-93e1-db261a697e3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c19ff91e06734ff4a115ac01c0f722fb', 'request_id': 'req-7a062070-d51a-4c86-93e1-db261a697e3e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:21.193070 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7a062070-d51a-4c86-93e1-db261a697e3e admin admin] The token cache is being invalidate because project c19ff91e06734ff4a115ac01c0f722fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:15:21.193443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7a062070-d51a-4c86-93e1-db261a697e3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:21.195273 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7a062070-d51a-4c86-93e1-db261a697e3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:21.197221 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 959/1890] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:15:20 2022] DELETE /identity/v3/projects/c19ff91e06734ff4a115ac01c0f722fb => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:21.202211 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a062070-d51a-4c86-93e1-db261a697e3e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:21.203751 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7a062070-d51a-4c86-93e1-db261a697e3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:21.258536 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ca84a2-308b-4454-8d91-5fce6887f16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:21.259263 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca84a2-308b-4454-8d91-5fce6887f16b admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:21.259548 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca84a2-308b-4454-8d91-5fce6887f16b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:21.259831 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ca84a2-308b-4454-8d91-5fce6887f16b admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:21.260534 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ca84a2-308b-4454-8d91-5fce6887f16b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:21.264096 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ca84a2-308b-4454-8d91-5fce6887f16b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:21.309439 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a062070-d51a-4c86-93e1-db261a697e3e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:21.316265 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642fd467-1f93-422e-bc08-43fbfe37e94e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:21.316946 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642fd467-1f93-422e-bc08-43fbfe37e94e service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:21.317236 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642fd467-1f93-422e-bc08-43fbfe37e94e service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:21.317517 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642fd467-1f93-422e-bc08-43fbfe37e94e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:21.321347 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-642fd467-1f93-422e-bc08-43fbfe37e94e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:21.321466 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 932/1891] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:15:21 2022] POST /identity/v3/projects => generated 387 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:21.326071 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ca84a2-308b-4454-8d91-5fce6887f16b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:21.327790 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c2ca84a2-308b-4454-8d91-5fce6887f16b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:21.343595 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642fd467-1f93-422e-bc08-43fbfe37e94e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:21.345201 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642fd467-1f93-422e-bc08-43fbfe37e94e service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:21.356018 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 960/1892] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:21.356896 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642fd467-1f93-422e-bc08-43fbfe37e94e service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:21.358153 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-642fd467-1f93-422e-bc08-43fbfe37e94e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:21.372495 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642fd467-1f93-422e-bc08-43fbfe37e94e service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:21.464328 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ca84a2-308b-4454-8d91-5fce6887f16b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:21.472735 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0089d1-ba06-4ab7-8eec-14bf7715fc5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:21.474174 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0089d1-ba06-4ab7-8eec-14bf7715fc5f admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:21.474428 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fc7079-6bf8-45f1-b6c9-eb110cbda94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:21.474597 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0089d1-ba06-4ab7-8eec-14bf7715fc5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:21.475277 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0089d1-ba06-4ab7-8eec-14bf7715fc5f admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:21.475708 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc7079-6bf8-45f1-b6c9-eb110cbda94e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:21.476061 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc7079-6bf8-45f1-b6c9-eb110cbda94e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:21.476171 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0089d1-ba06-4ab7-8eec-14bf7715fc5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:21.476545 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc7079-6bf8-45f1-b6c9-eb110cbda94e admin admin] PATH_INFO: `/v3/users/3b0fda599bb34e378d1a3f7cf66ba5e4` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:21.478088 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fc7079-6bf8-45f1-b6c9-eb110cbda94e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b0fda599bb34e378d1a3f7cf66ba5e4)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:21.478538 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0089d1-ba06-4ab7-8eec-14bf7715fc5f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:21.480568 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fc7079-6bf8-45f1-b6c9-eb110cbda94e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:21.509004 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 961/1893] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:15:21 2022] POST /identity/v3/users => generated 300 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:21.512778 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0089d1-ba06-4ab7-8eec-14bf7715fc5f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:21.525032 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0089d1-ba06-4ab7-8eec-14bf7715fc5f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:21.530902 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55238dc5-6109-4f99-b74f-c132f0d49802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:21.531191 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55238dc5-6109-4f99-b74f-c132f0d49802 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:21.531405 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55238dc5-6109-4f99-b74f-c132f0d49802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:21.531692 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55238dc5-6109-4f99-b74f-c132f0d49802 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:21.532582 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55238dc5-6109-4f99-b74f-c132f0d49802 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:21.534409 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55238dc5-6109-4f99-b74f-c132f0d49802 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:21.550837 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 962/1894] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:21.551686 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55238dc5-6109-4f99-b74f-c132f0d49802 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:21.563403 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-53fc7079-6bf8-45f1-b6c9-eb110cbda94e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b0fda599bb34e378d1a3f7cf66ba5e4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:21.564835 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55238dc5-6109-4f99-b74f-c132f0d49802 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:21.569733 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05415ae7-4005-4f5a-8af8-b55fe0821b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:21.571237 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05415ae7-4005-4f5a-8af8-b55fe0821b32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:21.571531 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05415ae7-4005-4f5a-8af8-b55fe0821b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:21.571835 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05415ae7-4005-4f5a-8af8-b55fe0821b32 admin admin] PATH_INFO: `/v3/projects/071100ed27e2483c9d0c7ca3b50f577e/users/81382ab8147d42bbbb4469773550f912/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:21.577738 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-53fc7079-6bf8-45f1-b6c9-eb110cbda94e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b0fda599bb34e378d1a3f7cf66ba5e4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:21.590446 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-53fc7079-6bf8-45f1-b6c9-eb110cbda94e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b0fda599bb34e378d1a3f7cf66ba5e4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:21.591290 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-53fc7079-6bf8-45f1-b6c9-eb110cbda94e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b0fda599bb34e378d1a3f7cf66ba5e4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:21.591976 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-53fc7079-6bf8-45f1-b6c9-eb110cbda94e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b0fda599bb34e378d1a3f7cf66ba5e4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:21.613605 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05415ae7-4005-4f5a-8af8-b55fe0821b32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=071100ed27e2483c9d0c7ca3b50f577e, user_id=81382ab8147d42bbbb4469773550f912, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:21.615861 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05415ae7-4005-4f5a-8af8-b55fe0821b32 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:21.627224 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 933/1895] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:15:21 2022] DELETE /identity/v3/users/3b0fda599bb34e378d1a3f7cf66ba5e4 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:21.649122 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 963/1896] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:21 2022] PUT /identity/v3/projects/071100ed27e2483c9d0c7ca3b50f577e/users/81382ab8147d42bbbb4469773550f912/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:21.650072 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fc7079-6bf8-45f1-b6c9-eb110cbda94e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:21.650645 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05415ae7-4005-4f5a-8af8-b55fe0821b32 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:21.652941 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-53fc7079-6bf8-45f1-b6c9-eb110cbda94e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:21.653314 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-05415ae7-4005-4f5a-8af8-b55fe0821b32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:21.675183 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:21.718949 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:DELETE=1,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:21.764362 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fc7079-6bf8-45f1-b6c9-eb110cbda94e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:21.766936 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05415ae7-4005-4f5a-8af8-b55fe0821b32 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:21.772982 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c65418-778a-45c0-b37a-71295faf47b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:21.775399 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c65418-778a-45c0-b37a-71295faf47b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:21.775892 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c65418-778a-45c0-b37a-71295faf47b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:21.776434 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c65418-778a-45c0-b37a-71295faf47b6 admin admin] PATH_INFO: `/v3/users/0b42cad9fde34170b9ad058e24e761e5` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:21.779810 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c65418-778a-45c0-b37a-71295faf47b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b42cad9fde34170b9ad058e24e761e5)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:21.781766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b616c-9118-408b-8712-a2b03723c9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:21.783657 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b616c-9118-408b-8712-a2b03723c9e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:21.783657 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b616c-9118-408b-8712-a2b03723c9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:21.786162 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b616c-9118-408b-8712-a2b03723c9e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:21.788717 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b616c-9118-408b-8712-a2b03723c9e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:21.789170 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c65418-778a-45c0-b37a-71295faf47b6 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:21.794346 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b616c-9118-408b-8712-a2b03723c9e7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:21.824272 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 964/1897] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:21.828400 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b616c-9118-408b-8712-a2b03723c9e7 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:21.844405 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b616c-9118-408b-8712-a2b03723c9e7 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:21.858194 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce67d255-8ada-4daf-9da5-6c925a941bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:21.860619 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67d255-8ada-4daf-9da5-6c925a941bcf admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:21.862655 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67d255-8ada-4daf-9da5-6c925a941bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:21.863677 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67d255-8ada-4daf-9da5-6c925a941bcf admin admin] PATH_INFO: `/v3/projects/071100ed27e2483c9d0c7ca3b50f577e/users/81382ab8147d42bbbb4469773550f912/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:21.867197 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce67d255-8ada-4daf-9da5-6c925a941bcf admin admin] RBAC: Authorizing `identity:create_grant(project_id=071100ed27e2483c9d0c7ca3b50f577e, user_id=81382ab8147d42bbbb4469773550f912, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:21.875347 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce67d255-8ada-4daf-9da5-6c925a941bcf admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:21.887683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-91c65418-778a-45c0-b37a-71295faf47b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b42cad9fde34170b9ad058e24e761e5'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:21.904813 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-91c65418-778a-45c0-b37a-71295faf47b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b42cad9fde34170b9ad058e24e761e5'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:21.920238 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-91c65418-778a-45c0-b37a-71295faf47b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b42cad9fde34170b9ad058e24e761e5'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:21.921185 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-91c65418-778a-45c0-b37a-71295faf47b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b42cad9fde34170b9ad058e24e761e5'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:21.922033 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-91c65418-778a-45c0-b37a-71295faf47b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b42cad9fde34170b9ad058e24e761e5'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:21.927971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 965/1898] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:21 2022] PUT /identity/v3/projects/071100ed27e2483c9d0c7ca3b50f577e/users/81382ab8147d42bbbb4469773550f912/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:15:21.940861 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7abc4-821b-407b-8210-5a63007baa15 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:21.941250 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7abc4-821b-407b-8210-5a63007baa15 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:21.941692 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c7abc4-821b-407b-8210-5a63007baa15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:21.948293 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 934/1899] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:15:21 2022] DELETE /identity/v3/users/0b42cad9fde34170b9ad058e24e761e5 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:21.953428 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed51b8-f4ae-4e0b-a893-b90191ef0b78 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:21.954440 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed51b8-f4ae-4e0b-a893-b90191ef0b78 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:21.954945 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed51b8-f4ae-4e0b-a893-b90191ef0b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:21.982693 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-66ed51b8-f4ae-4e0b-a893-b90191ef0b78 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:21.983975 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-66ed51b8-f4ae-4e0b-a893-b90191ef0b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:21.985656 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-07c7abc4-821b-407b-8210-5a63007baa15 None None] MFA Rules not processed for user `81382ab8147d42bbbb4469773550f912`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:21.987133 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-07c7abc4-821b-407b-8210-5a63007baa15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:22.095961 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 935/1900] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:22.097026 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed51b8-f4ae-4e0b-a893-b90191ef0b78 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:22.099354 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-66ed51b8-f4ae-4e0b-a893-b90191ef0b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:22.128040 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 966/1901] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:21 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:22.128940 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c7abc4-821b-407b-8210-5a63007baa15 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:22.130550 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed51b8-f4ae-4e0b-a893-b90191ef0b78 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:22.134804 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-07c7abc4-821b-407b-8210-5a63007baa15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:22.143690 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497ea9fd-0936-46ca-b8f0-d620491a3dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:22.144581 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ea9fd-0936-46ca-b8f0-d620491a3dd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:22.144868 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ea9fd-0936-46ca-b8f0-d620491a3dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:22.145149 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ea9fd-0936-46ca-b8f0-d620491a3dd7 admin admin] PATH_INFO: `/v3/projects/097fe7f4f0e844b98f1254a343a6f24f` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:22.146404 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497ea9fd-0936-46ca-b8f0-d620491a3dd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=097fe7f4f0e844b98f1254a343a6f24f)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:22.148392 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497ea9fd-0936-46ca-b8f0-d620491a3dd7 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:22.187232 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:22.210812 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-497ea9fd-0936-46ca-b8f0-d620491a3dd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '097fe7f4f0e844b98f1254a343a6f24f', 'request_id': 'req-497ea9fd-0936-46ca-b8f0-d620491a3dd7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:22.219047 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-497ea9fd-0936-46ca-b8f0-d620491a3dd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '097fe7f4f0e844b98f1254a343a6f24f', 'request_id': 'req-497ea9fd-0936-46ca-b8f0-d620491a3dd7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:22.226757 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-497ea9fd-0936-46ca-b8f0-d620491a3dd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '097fe7f4f0e844b98f1254a343a6f24f', 'request_id': 'req-497ea9fd-0936-46ca-b8f0-d620491a3dd7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:22.234798 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c7abc4-821b-407b-8210-5a63007baa15 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:22.251215 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-497ea9fd-0936-46ca-b8f0-d620491a3dd7 admin admin] The token cache is being invalidate because project 097fe7f4f0e844b98f1254a343a6f24f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:15:22.251873 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-497ea9fd-0936-46ca-b8f0-d620491a3dd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:22.252722 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-497ea9fd-0936-46ca-b8f0-d620491a3dd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:22.254835 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 936/1902] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:15:22 2022] DELETE /identity/v3/projects/097fe7f4f0e844b98f1254a343a6f24f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:22.259997 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497ea9fd-0936-46ca-b8f0-d620491a3dd7 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:22.263202 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-497ea9fd-0936-46ca-b8f0-d620491a3dd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:22.301786 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb395049-11d8-4be2-b2d4-1e3dd24cb543 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:22.302926 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb395049-11d8-4be2-b2d4-1e3dd24cb543 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:22.302926 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb395049-11d8-4be2-b2d4-1e3dd24cb543 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:22.303391 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb395049-11d8-4be2-b2d4-1e3dd24cb543 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:22.305391 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fb395049-11d8-4be2-b2d4-1e3dd24cb543 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:22.327396 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497ea9fd-0936-46ca-b8f0-d620491a3dd7 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:22.331016 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422d3de0-38eb-49c5-9aa7-90b6627bff10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:22.331911 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d3de0-38eb-49c5-9aa7-90b6627bff10 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:22.332019 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:22.332459 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d3de0-38eb-49c5-9aa7-90b6627bff10 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:22.332459 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422d3de0-38eb-49c5-9aa7-90b6627bff10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:22.334245 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-422d3de0-38eb-49c5-9aa7-90b6627bff10 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:22.377781 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb395049-11d8-4be2-b2d4-1e3dd24cb543 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:22.379367 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb395049-11d8-4be2-b2d4-1e3dd24cb543 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:22.390418 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 967/1903] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:22.391409 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb395049-11d8-4be2-b2d4-1e3dd24cb543 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:22.392872 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fb395049-11d8-4be2-b2d4-1e3dd24cb543 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:22.404738 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb395049-11d8-4be2-b2d4-1e3dd24cb543 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:22.405231 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422d3de0-38eb-49c5-9aa7-90b6627bff10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:22.406882 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422d3de0-38eb-49c5-9aa7-90b6627bff10 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:22.409468 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ca01b1-096e-4302-9417-6f52e23c7f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:22.410337 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca01b1-096e-4302-9417-6f52e23c7f7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:22.410618 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca01b1-096e-4302-9417-6f52e23c7f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:22.410916 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca01b1-096e-4302-9417-6f52e23c7f7a admin admin] PATH_INFO: `/v3/projects/a2f2efa98746466ba9c45defd8bb8183` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:22.412086 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ca01b1-096e-4302-9417-6f52e23c7f7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2f2efa98746466ba9c45defd8bb8183)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:22.413975 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ca01b1-096e-4302-9417-6f52e23c7f7a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:22.423489 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 937/1904] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:22 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:22.424745 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f9161a-97ee-4052-87eb-86ee6d775cec None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:22.425029 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f9161a-97ee-4052-87eb-86ee6d775cec None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:22.427455 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f9161a-97ee-4052-87eb-86ee6d775cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:22.464496 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-39f9161a-97ee-4052-87eb-86ee6d775cec None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:22.466690 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-39f9161a-97ee-4052-87eb-86ee6d775cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:22.480774 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 938/1905] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:22.491188 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19993ba-3eac-48d0-9204-317c87295741 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:22.491521 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19993ba-3eac-48d0-9204-317c87295741 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:22.491753 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19993ba-3eac-48d0-9204-317c87295741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:22.522477 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c4ca01b1-096e-4302-9417-6f52e23c7f7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2f2efa98746466ba9c45defd8bb8183', 'request_id': 'req-c4ca01b1-096e-4302-9417-6f52e23c7f7a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:22.526771 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f19993ba-3eac-48d0-9204-317c87295741 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:22.528985 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f19993ba-3eac-48d0-9204-317c87295741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:22.536947 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c4ca01b1-096e-4302-9417-6f52e23c7f7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2f2efa98746466ba9c45defd8bb8183', 'request_id': 'req-c4ca01b1-096e-4302-9417-6f52e23c7f7a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:22.546278 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c4ca01b1-096e-4302-9417-6f52e23c7f7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2f2efa98746466ba9c45defd8bb8183', 'request_id': 'req-c4ca01b1-096e-4302-9417-6f52e23c7f7a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:22.584253 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c4ca01b1-096e-4302-9417-6f52e23c7f7a admin admin] The token cache is being invalidate because project a2f2efa98746466ba9c45defd8bb8183 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:15:22.584549 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c4ca01b1-096e-4302-9417-6f52e23c7f7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:22.585840 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c4ca01b1-096e-4302-9417-6f52e23c7f7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:22.594793 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 968/1906] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:15:22 2022] DELETE /identity/v3/projects/a2f2efa98746466ba9c45defd8bb8183 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:22.596358 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ca01b1-096e-4302-9417-6f52e23c7f7a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:22.597848 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c4ca01b1-096e-4302-9417-6f52e23c7f7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:22.651829 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:22.667932 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 939/1907] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:22.668807 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19993ba-3eac-48d0-9204-317c87295741 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:22.670132 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f19993ba-3eac-48d0-9204-317c87295741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:22.682944 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19993ba-3eac-48d0-9204-317c87295741 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:22.687226 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ca01b1-096e-4302-9417-6f52e23c7f7a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:22.688211 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e498db-4274-4716-8350-092088de23e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:22.689085 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e498db-4274-4716-8350-092088de23e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:22.689371 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e498db-4274-4716-8350-092088de23e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:22.689645 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e498db-4274-4716-8350-092088de23e8 admin admin] PATH_INFO: `/v3/users/d1580f3882134b7dbcbfbf736771dfd6` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:22.690840 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e498db-4274-4716-8350-092088de23e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1580f3882134b7dbcbfbf736771dfd6)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:22.692856 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e498db-4274-4716-8350-092088de23e8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:22.692967 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af502c07-bab7-4280-a0ba-108f824aff53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:22.693764 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af502c07-bab7-4280-a0ba-108f824aff53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:22.694048 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af502c07-bab7-4280-a0ba-108f824aff53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:22.694326 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af502c07-bab7-4280-a0ba-108f824aff53 admin admin] PATH_INFO: `/v3/users/f81db5617339444f80e45610642d9cdf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:22.695571 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af502c07-bab7-4280-a0ba-108f824aff53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f81db5617339444f80e45610642d9cdf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:22.698074 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af502c07-bab7-4280-a0ba-108f824aff53 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:22.748381 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-51e498db-4274-4716-8350-092088de23e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1580f3882134b7dbcbfbf736771dfd6'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:22.749392 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:22.755888 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-af502c07-bab7-4280-a0ba-108f824aff53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f81db5617339444f80e45610642d9cdf'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:22.759939 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-51e498db-4274-4716-8350-092088de23e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1580f3882134b7dbcbfbf736771dfd6'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:22.765458 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-af502c07-bab7-4280-a0ba-108f824aff53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f81db5617339444f80e45610642d9cdf'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:22.766342 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-af502c07-bab7-4280-a0ba-108f824aff53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f81db5617339444f80e45610642d9cdf'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:22.771856 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-51e498db-4274-4716-8350-092088de23e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1580f3882134b7dbcbfbf736771dfd6'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:22.773170 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-51e498db-4274-4716-8350-092088de23e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1580f3882134b7dbcbfbf736771dfd6'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:22.774365 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-51e498db-4274-4716-8350-092088de23e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1580f3882134b7dbcbfbf736771dfd6'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:22.775540 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-af502c07-bab7-4280-a0ba-108f824aff53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f81db5617339444f80e45610642d9cdf'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:22.776858 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-af502c07-bab7-4280-a0ba-108f824aff53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f81db5617339444f80e45610642d9cdf'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:22.800476 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 969/1908] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:15:22 2022] DELETE /identity/v3/users/f81db5617339444f80e45610642d9cdf => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:22.803483 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af502c07-bab7-4280-a0ba-108f824aff53 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:22.803483 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-af502c07-bab7-4280-a0ba-108f824aff53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:22.809287 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 940/1909] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:15:22 2022] DELETE /identity/v3/users/d1580f3882134b7dbcbfbf736771dfd6 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:22.809287 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e498db-4274-4716-8350-092088de23e8 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:22.809287 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-51e498db-4274-4716-8350-092088de23e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:22.911975 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af502c07-bab7-4280-a0ba-108f824aff53 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:22.917808 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e498db-4274-4716-8350-092088de23e8 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:22.919072 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c977113-af96-463e-9c1a-88ee9a2304bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:22.920074 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c977113-af96-463e-9c1a-88ee9a2304bc admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:22.920392 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c977113-af96-463e-9c1a-88ee9a2304bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:22.920669 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c977113-af96-463e-9c1a-88ee9a2304bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:22.921991 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c977113-af96-463e-9c1a-88ee9a2304bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:15:22.922414 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c977113-af96-463e-9c1a-88ee9a2304bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:22.925670 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c977113-af96-463e-9c1a-88ee9a2304bc admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:22.932529 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c0af1e-c307-468e-ae96-955484b84c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:22.933434 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0af1e-c307-468e-ae96-955484b84c56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:22.935837 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0af1e-c307-468e-ae96-955484b84c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:22.936178 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0af1e-c307-468e-ae96-955484b84c56 admin admin] PATH_INFO: `/v3/projects/59c6c742cd434430a8e7b7e1fa954335` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:22.946833 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 970/1910] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:15:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:15:22.947045 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c0af1e-c307-468e-ae96-955484b84c56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59c6c742cd434430a8e7b7e1fa954335)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:22.949011 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d38b7-2545-488f-937d-20363552cc09 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:22.949777 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c0af1e-c307-468e-ae96-955484b84c56 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:22.949888 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d38b7-2545-488f-937d-20363552cc09 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:22.950111 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d38b7-2545-488f-937d-20363552cc09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:22.978022 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d53d38b7-2545-488f-937d-20363552cc09 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:22.987230 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d53d38b7-2545-488f-937d-20363552cc09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:23.003365 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-28c0af1e-c307-468e-ae96-955484b84c56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59c6c742cd434430a8e7b7e1fa954335', 'request_id': 'req-28c0af1e-c307-468e-ae96-955484b84c56'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:23.009100 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-28c0af1e-c307-468e-ae96-955484b84c56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59c6c742cd434430a8e7b7e1fa954335', 'request_id': 'req-28c0af1e-c307-468e-ae96-955484b84c56'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:23.016288 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-28c0af1e-c307-468e-ae96-955484b84c56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59c6c742cd434430a8e7b7e1fa954335', 'request_id': 'req-28c0af1e-c307-468e-ae96-955484b84c56'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:23.028691 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:23.033646 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-28c0af1e-c307-468e-ae96-955484b84c56 admin admin] The token cache is being invalidate because project 59c6c742cd434430a8e7b7e1fa954335 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:15:23.034345 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-28c0af1e-c307-468e-ae96-955484b84c56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:23.035774 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-28c0af1e-c307-468e-ae96-955484b84c56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:23.037563 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 941/1911] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:15:22 2022] DELETE /identity/v3/projects/59c6c742cd434430a8e7b7e1fa954335 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:23.038744 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c0af1e-c307-468e-ae96-955484b84c56 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:23.042877 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-28c0af1e-c307-468e-ae96-955484b84c56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:23.088221 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 971/1912] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:23.091925 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d38b7-2545-488f-937d-20363552cc09 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:23.096465 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d53d38b7-2545-488f-937d-20363552cc09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:23.111146 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d38b7-2545-488f-937d-20363552cc09 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:23.117909 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa50c44-d7fc-40d8-b880-39b456654606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:23.118800 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa50c44-d7fc-40d8-b880-39b456654606 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:23.119122 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa50c44-d7fc-40d8-b880-39b456654606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:23.119403 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa50c44-d7fc-40d8-b880-39b456654606 admin admin] PATH_INFO: `/v3/users/b179bcde825a465ab113c2fe2bc62661` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:23.121064 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa50c44-d7fc-40d8-b880-39b456654606 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b179bcde825a465ab113c2fe2bc62661)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:23.123039 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa50c44-d7fc-40d8-b880-39b456654606 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:23.130996 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c0af1e-c307-468e-ae96-955484b84c56 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:23.138368 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc60c883-a1cd-4fdc-9cee-1c07e293dffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:23.139155 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc60c883-a1cd-4fdc-9cee-1c07e293dffa admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:23.139898 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc60c883-a1cd-4fdc-9cee-1c07e293dffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:23.139898 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc60c883-a1cd-4fdc-9cee-1c07e293dffa admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:23.140639 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc60c883-a1cd-4fdc-9cee-1c07e293dffa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:23.142626 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc60c883-a1cd-4fdc-9cee-1c07e293dffa admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:23.185076 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 942/1913] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:15:23 2022] POST /identity/v3/projects => generated 385 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:23.185076 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc60c883-a1cd-4fdc-9cee-1c07e293dffa admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:23.185482 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bc60c883-a1cd-4fdc-9cee-1c07e293dffa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:23.213076 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9aa50c44-d7fc-40d8-b880-39b456654606 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b179bcde825a465ab113c2fe2bc62661'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:23.218771 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:23.240730 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9aa50c44-d7fc-40d8-b880-39b456654606 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b179bcde825a465ab113c2fe2bc62661'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:23.241844 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9aa50c44-d7fc-40d8-b880-39b456654606 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b179bcde825a465ab113c2fe2bc62661'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:23.253421 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9aa50c44-d7fc-40d8-b880-39b456654606 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b179bcde825a465ab113c2fe2bc62661'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:23.254536 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9aa50c44-d7fc-40d8-b880-39b456654606 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b179bcde825a465ab113c2fe2bc62661'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:23.287747 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 972/1914] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:15:23 2022] DELETE /identity/v3/users/b179bcde825a465ab113c2fe2bc62661 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:23.288605 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa50c44-d7fc-40d8-b880-39b456654606 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:23.291492 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9aa50c44-d7fc-40d8-b880-39b456654606 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:23.304776 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc60c883-a1cd-4fdc-9cee-1c07e293dffa admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:23.414747 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7db3c4-033e-49f9-958d-8e54b6e85dca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:23.415485 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7db3c4-033e-49f9-958d-8e54b6e85dca service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:23.415804 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7db3c4-033e-49f9-958d-8e54b6e85dca service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:23.416156 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7db3c4-033e-49f9-958d-8e54b6e85dca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:23.417808 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa50c44-d7fc-40d8-b880-39b456654606 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:23.420811 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2d7db3c4-033e-49f9-958d-8e54b6e85dca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:23.426354 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5c0e44-0722-4637-96c9-33e1c6cc56ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:23.427575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c0e44-0722-4637-96c9-33e1c6cc56ee admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:23.427875 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c0e44-0722-4637-96c9-33e1c6cc56ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:23.428153 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c0e44-0722-4637-96c9-33e1c6cc56ee admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:23.429364 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5c0e44-0722-4637-96c9-33e1c6cc56ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:23.431796 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5c0e44-0722-4637-96c9-33e1c6cc56ee admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:23.438347 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7db3c4-033e-49f9-958d-8e54b6e85dca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:23.440539 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7db3c4-033e-49f9-958d-8e54b6e85dca service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:23.454649 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 943/1915] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:23.456219 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7db3c4-033e-49f9-958d-8e54b6e85dca service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:23.457650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2d7db3c4-033e-49f9-958d-8e54b6e85dca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:23.460804 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 973/1916] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:15:23 2022] POST /identity/v3/users => generated 293 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:23.461889 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5c0e44-0722-4637-96c9-33e1c6cc56ee admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:23.472237 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7db3c4-033e-49f9-958d-8e54b6e85dca service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:23.474970 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5c0e44-0722-4637-96c9-33e1c6cc56ee admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:23.479174 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957d7f5d-c4dd-4b2c-94ca-460030aba5e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:23.480822 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d7f5d-c4dd-4b2c-94ca-460030aba5e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:23.481305 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d7f5d-c4dd-4b2c-94ca-460030aba5e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:23.481684 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957d7f5d-c4dd-4b2c-94ca-460030aba5e4 admin admin] PATH_INFO: `/v3/projects/6e9a4f4c71054f5391a2fc79b5121292` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:23.482178 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ca19e-734d-4400-9cb2-c4df357617bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:23.483053 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ca19e-734d-4400-9cb2-c4df357617bb admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:23.483255 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957d7f5d-c4dd-4b2c-94ca-460030aba5e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e9a4f4c71054f5391a2fc79b5121292)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:23.483644 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ca19e-734d-4400-9cb2-c4df357617bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:23.484060 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ca19e-734d-4400-9cb2-c4df357617bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:23.484923 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551ca19e-734d-4400-9cb2-c4df357617bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:23.485967 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957d7f5d-c4dd-4b2c-94ca-460030aba5e4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:23.486891 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551ca19e-734d-4400-9cb2-c4df357617bb admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:23.503955 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 974/1917] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:23 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:23.505663 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ca19e-734d-4400-9cb2-c4df357617bb admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:23.521938 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ca19e-734d-4400-9cb2-c4df357617bb admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:23.528557 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75145457-15c0-47df-97bd-beaa17332539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:23.529779 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75145457-15c0-47df-97bd-beaa17332539 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:23.530288 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75145457-15c0-47df-97bd-beaa17332539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:23.530702 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75145457-15c0-47df-97bd-beaa17332539 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:23.532059 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75145457-15c0-47df-97bd-beaa17332539 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:23.533720 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75145457-15c0-47df-97bd-beaa17332539 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:23.547262 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 975/1918] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:23 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:23.551661 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75145457-15c0-47df-97bd-beaa17332539 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:23.556692 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:23.569222 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75145457-15c0-47df-97bd-beaa17332539 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:23.571528 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-957d7f5d-c4dd-4b2c-94ca-460030aba5e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e9a4f4c71054f5391a2fc79b5121292', 'request_id': 'req-957d7f5d-c4dd-4b2c-94ca-460030aba5e4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:23.582885 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-957d7f5d-c4dd-4b2c-94ca-460030aba5e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e9a4f4c71054f5391a2fc79b5121292', 'request_id': 'req-957d7f5d-c4dd-4b2c-94ca-460030aba5e4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:23.592447 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-957d7f5d-c4dd-4b2c-94ca-460030aba5e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e9a4f4c71054f5391a2fc79b5121292', 'request_id': 'req-957d7f5d-c4dd-4b2c-94ca-460030aba5e4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:23.613163 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-957d7f5d-c4dd-4b2c-94ca-460030aba5e4 admin admin] The token cache is being invalidate because project 6e9a4f4c71054f5391a2fc79b5121292 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:15:23.614141 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-957d7f5d-c4dd-4b2c-94ca-460030aba5e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:23.615043 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-957d7f5d-c4dd-4b2c-94ca-460030aba5e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:23.617695 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 944/1919] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:15:23 2022] DELETE /identity/v3/projects/6e9a4f4c71054f5391a2fc79b5121292 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:23.623167 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd3c6b-369b-4ddf-9d18-b13ec51a725d None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:23.623464 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd3c6b-369b-4ddf-9d18-b13ec51a725d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:23.623880 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fd3c6b-369b-4ddf-9d18-b13ec51a725d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:23.681021 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-48fd3c6b-369b-4ddf-9d18-b13ec51a725d None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:23.684777 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-48fd3c6b-369b-4ddf-9d18-b13ec51a725d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:23.700595 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872cfe61-3e3f-44b9-8327-42cf0607a0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:23.705443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872cfe61-3e3f-44b9-8327-42cf0607a0f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:23.705738 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872cfe61-3e3f-44b9-8327-42cf0607a0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:23.706020 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872cfe61-3e3f-44b9-8327-42cf0607a0f4 admin admin] PATH_INFO: `/v3/projects/bc5be122b10744a09a7397129532ea26/users/a50c6cab092b47329c9f5802168ca355/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:23.765245 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872cfe61-3e3f-44b9-8327-42cf0607a0f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc5be122b10744a09a7397129532ea26, user_id=a50c6cab092b47329c9f5802168ca355, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:23.773075 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872cfe61-3e3f-44b9-8327-42cf0607a0f4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:23.817416 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 976/1920] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:23 2022] PUT /identity/v3/projects/bc5be122b10744a09a7397129532ea26/users/a50c6cab092b47329c9f5802168ca355/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:15:23.819817 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872cfe61-3e3f-44b9-8327-42cf0607a0f4 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:23.831661 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-872cfe61-3e3f-44b9-8327-42cf0607a0f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:23.867977 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 945/1921] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:23.871545 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b678053f-0242-486a-999e-cd56b44e79aa None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:23.873004 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b678053f-0242-486a-999e-cd56b44e79aa None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:23.873410 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b678053f-0242-486a-999e-cd56b44e79aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:23.890908 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:23.941198 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b678053f-0242-486a-999e-cd56b44e79aa None None] MFA Rules not processed for user `a50c6cab092b47329c9f5802168ca355`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:23.942802 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b678053f-0242-486a-999e-cd56b44e79aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:23.990700 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872cfe61-3e3f-44b9-8327-42cf0607a0f4 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:24.015561 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bceecd4-2152-4e67-b5bd-adc9f32ca037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:24.017920 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bceecd4-2152-4e67-b5bd-adc9f32ca037 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:24.017920 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bceecd4-2152-4e67-b5bd-adc9f32ca037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:24.017920 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bceecd4-2152-4e67-b5bd-adc9f32ca037 admin admin] PATH_INFO: `/v3/projects/d7439103a9514268ab6be38eb1860b7f` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:24.019108 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bceecd4-2152-4e67-b5bd-adc9f32ca037 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7439103a9514268ab6be38eb1860b7f)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:24.021128 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bceecd4-2152-4e67-b5bd-adc9f32ca037 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:24.046706 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 946/1922] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:24.048291 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c8858-e672-4e58-929e-345d4e0ddad9 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:24.048501 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c8858-e672-4e58-929e-345d4e0ddad9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:24.048834 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c8858-e672-4e58-929e-345d4e0ddad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:24.070478 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4d0c8858-e672-4e58-929e-345d4e0ddad9 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:24.071720 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4d0c8858-e672-4e58-929e-345d4e0ddad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:24.083194 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3bceecd4-2152-4e67-b5bd-adc9f32ca037 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7439103a9514268ab6be38eb1860b7f', 'request_id': 'req-3bceecd4-2152-4e67-b5bd-adc9f32ca037'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:24.092055 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3bceecd4-2152-4e67-b5bd-adc9f32ca037 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7439103a9514268ab6be38eb1860b7f', 'request_id': 'req-3bceecd4-2152-4e67-b5bd-adc9f32ca037'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:24.100544 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3bceecd4-2152-4e67-b5bd-adc9f32ca037 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7439103a9514268ab6be38eb1860b7f', 'request_id': 'req-3bceecd4-2152-4e67-b5bd-adc9f32ca037'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:24.126570 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3bceecd4-2152-4e67-b5bd-adc9f32ca037 admin admin] The token cache is being invalidate because project d7439103a9514268ab6be38eb1860b7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:15:24.127029 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3bceecd4-2152-4e67-b5bd-adc9f32ca037 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:24.127832 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3bceecd4-2152-4e67-b5bd-adc9f32ca037 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:24.128517 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:24.129558 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 977/1923] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:15:23 2022] DELETE /identity/v3/projects/d7439103a9514268ab6be38eb1860b7f => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:24.130801 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bceecd4-2152-4e67-b5bd-adc9f32ca037 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:24.132357 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3bceecd4-2152-4e67-b5bd-adc9f32ca037 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:24.161497 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bceecd4-2152-4e67-b5bd-adc9f32ca037 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:24.163583 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 947/1924] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:24.167603 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c8858-e672-4e58-929e-345d4e0ddad9 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:24.172414 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4d0c8858-e672-4e58-929e-345d4e0ddad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:24.176605 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02814d6e-117d-47fb-a41f-195028c8e6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:24.178692 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02814d6e-117d-47fb-a41f-195028c8e6bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:24.180903 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02814d6e-117d-47fb-a41f-195028c8e6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:24.180903 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02814d6e-117d-47fb-a41f-195028c8e6bd admin admin] PATH_INFO: `/v3/projects/99da31ee7fe74f2bbd6789f4e2a3b696` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:24.183957 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02814d6e-117d-47fb-a41f-195028c8e6bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=99da31ee7fe74f2bbd6789f4e2a3b696)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:24.187674 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02814d6e-117d-47fb-a41f-195028c8e6bd admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:24.190419 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c8858-e672-4e58-929e-345d4e0ddad9 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:24.195668 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef36b07-4edf-49a3-add1-6ec08f5a7317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:24.196450 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef36b07-4edf-49a3-add1-6ec08f5a7317 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:24.196839 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef36b07-4edf-49a3-add1-6ec08f5a7317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:24.197218 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef36b07-4edf-49a3-add1-6ec08f5a7317 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:24.198018 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef36b07-4edf-49a3-add1-6ec08f5a7317 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:24.200920 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef36b07-4edf-49a3-add1-6ec08f5a7317 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:24.252614 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 948/1925] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:15:24 2022] POST /identity/v3/projects => generated 387 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:24.254169 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-02814d6e-117d-47fb-a41f-195028c8e6bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99da31ee7fe74f2bbd6789f4e2a3b696', 'request_id': 'req-02814d6e-117d-47fb-a41f-195028c8e6bd'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:24.254580 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74427c-3db3-4e5b-86ac-fd51cc969014 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:24.255020 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74427c-3db3-4e5b-86ac-fd51cc969014 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:24.255691 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74427c-3db3-4e5b-86ac-fd51cc969014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:24.264516 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-02814d6e-117d-47fb-a41f-195028c8e6bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99da31ee7fe74f2bbd6789f4e2a3b696', 'request_id': 'req-02814d6e-117d-47fb-a41f-195028c8e6bd'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:24.272139 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-02814d6e-117d-47fb-a41f-195028c8e6bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99da31ee7fe74f2bbd6789f4e2a3b696', 'request_id': 'req-02814d6e-117d-47fb-a41f-195028c8e6bd'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:24.284030 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7e74427c-3db3-4e5b-86ac-fd51cc969014 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:24.285864 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7e74427c-3db3-4e5b-86ac-fd51cc969014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:24.288720 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-02814d6e-117d-47fb-a41f-195028c8e6bd admin admin] The token cache is being invalidate because project 99da31ee7fe74f2bbd6789f4e2a3b696 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:15:24.289014 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-02814d6e-117d-47fb-a41f-195028c8e6bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:24.289694 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-02814d6e-117d-47fb-a41f-195028c8e6bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:24.292267 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 978/1926] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:15:24 2022] DELETE /identity/v3/projects/99da31ee7fe74f2bbd6789f4e2a3b696 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:24.293172 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02814d6e-117d-47fb-a41f-195028c8e6bd admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:24.294546 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-02814d6e-117d-47fb-a41f-195028c8e6bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:24.349583 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:24.359444 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 949/1927] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:24.368087 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74427c-3db3-4e5b-86ac-fd51cc969014 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:24.369491 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7e74427c-3db3-4e5b-86ac-fd51cc969014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:24.384727 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74427c-3db3-4e5b-86ac-fd51cc969014 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:24.390637 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02814d6e-117d-47fb-a41f-195028c8e6bd admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:24.395582 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f1f3a-1861-480c-b277-d39d09a228fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:24.396314 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f1f3a-1861-480c-b277-d39d09a228fd admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:24.396613 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f1f3a-1861-480c-b277-d39d09a228fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:24.396836 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f1f3a-1861-480c-b277-d39d09a228fd admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:24.397540 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00f1f3a-1861-480c-b277-d39d09a228fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:24.399584 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00f1f3a-1861-480c-b277-d39d09a228fd admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:24.401691 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da773f9-2264-410e-8a0d-d8a79ea4b1ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:24.402619 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da773f9-2264-410e-8a0d-d8a79ea4b1ee admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:24.403167 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da773f9-2264-410e-8a0d-d8a79ea4b1ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:24.403544 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da773f9-2264-410e-8a0d-d8a79ea4b1ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:24.406135 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da773f9-2264-410e-8a0d-d8a79ea4b1ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:15:24.406603 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da773f9-2264-410e-8a0d-d8a79ea4b1ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:24.408836 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da773f9-2264-410e-8a0d-d8a79ea4b1ee admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:24.423465 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 979/1928] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:15:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:15:24.424386 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da773f9-2264-410e-8a0d-d8a79ea4b1ee admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:24.426185 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1da773f9-2264-410e-8a0d-d8a79ea4b1ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:24.430918 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 950/1929] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:15:24 2022] POST /identity/v3/users => generated 300 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:24.432362 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6da64b-398f-4660-ab7a-cfbc91298202 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:24.433394 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6da64b-398f-4660-ab7a-cfbc91298202 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:24.433898 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6da64b-398f-4660-ab7a-cfbc91298202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:24.442699 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da773f9-2264-410e-8a0d-d8a79ea4b1ee admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:24.450663 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596d77dc-3a53-431a-a6b1-344ebdac86be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:24.451727 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d77dc-3a53-431a-a6b1-344ebdac86be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:24.452083 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d77dc-3a53-431a-a6b1-344ebdac86be admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:24.452432 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d77dc-3a53-431a-a6b1-344ebdac86be admin admin] PATH_INFO: `/v3/users/81382ab8147d42bbbb4469773550f912` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:24.453528 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-cc6da64b-398f-4660-ab7a-cfbc91298202 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:24.453796 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596d77dc-3a53-431a-a6b1-344ebdac86be admin admin] RBAC: Authorizing `identity:delete_user(user_id=81382ab8147d42bbbb4469773550f912)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:24.454753 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cc6da64b-398f-4660-ab7a-cfbc91298202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:24.456145 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596d77dc-3a53-431a-a6b1-344ebdac86be admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:24.463160 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 951/1930] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:24.463930 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6da64b-398f-4660-ab7a-cfbc91298202 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:24.474567 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6da64b-398f-4660-ab7a-cfbc91298202 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:24.481414 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a929c5c-26b4-44ec-a760-c7f2a2549c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:24.482084 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a929c5c-26b4-44ec-a760-c7f2a2549c3a admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:24.482371 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a929c5c-26b4-44ec-a760-c7f2a2549c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:24.482649 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a929c5c-26b4-44ec-a760-c7f2a2549c3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:24.483389 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a929c5c-26b4-44ec-a760-c7f2a2549c3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:24.485722 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a929c5c-26b4-44ec-a760-c7f2a2549c3a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:24.513922 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-596d77dc-3a53-431a-a6b1-344ebdac86be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81382ab8147d42bbbb4469773550f912'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:24.516397 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 952/1931] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:15:24 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:24.517073 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a929c5c-26b4-44ec-a760-c7f2a2549c3a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:24.521889 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-596d77dc-3a53-431a-a6b1-344ebdac86be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81382ab8147d42bbbb4469773550f912'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:24.522593 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-596d77dc-3a53-431a-a6b1-344ebdac86be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81382ab8147d42bbbb4469773550f912'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:24.530649 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-596d77dc-3a53-431a-a6b1-344ebdac86be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81382ab8147d42bbbb4469773550f912'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:24.531416 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-596d77dc-3a53-431a-a6b1-344ebdac86be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81382ab8147d42bbbb4469773550f912'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:24.549915 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 980/1932] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:15:24 2022] DELETE /identity/v3/users/81382ab8147d42bbbb4469773550f912 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:24.551692 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596d77dc-3a53-431a-a6b1-344ebdac86be admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:24.553999 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-596d77dc-3a53-431a-a6b1-344ebdac86be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:24.595872 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a929c5c-26b4-44ec-a760-c7f2a2549c3a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:24.639137 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:24.639990 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596d77dc-3a53-431a-a6b1-344ebdac86be admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:24.645955 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae501b-7224-4aea-9866-ce48c9174603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:24.646713 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae501b-7224-4aea-9866-ce48c9174603 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:24.647198 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae501b-7224-4aea-9866-ce48c9174603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:24.647506 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbae501b-7224-4aea-9866-ce48c9174603 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:24.648495 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae501b-7224-4aea-9866-ce48c9174603 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:15:24.648808 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae501b-7224-4aea-9866-ce48c9174603 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:24.651009 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbae501b-7224-4aea-9866-ce48c9174603 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:24.667252 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 981/1933] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:15:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:15:24.668174 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae501b-7224-4aea-9866-ce48c9174603 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:24.669852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bbae501b-7224-4aea-9866-ce48c9174603 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:24.672165 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b193bcf3-5d62-4f5e-82e1-b1c4e02a61df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:24.672939 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b193bcf3-5d62-4f5e-82e1-b1c4e02a61df admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:24.673290 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b193bcf3-5d62-4f5e-82e1-b1c4e02a61df admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:24.673638 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b193bcf3-5d62-4f5e-82e1-b1c4e02a61df admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:24.674438 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b193bcf3-5d62-4f5e-82e1-b1c4e02a61df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:24.676188 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b193bcf3-5d62-4f5e-82e1-b1c4e02a61df admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:24.684562 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbae501b-7224-4aea-9866-ce48c9174603 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:24.687473 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 953/1934] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:24.688435 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b193bcf3-5d62-4f5e-82e1-b1c4e02a61df admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:24.693377 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50ea9f-f009-4041-a38f-829a9a1f2819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:24.694059 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50ea9f-f009-4041-a38f-829a9a1f2819 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:24.694383 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50ea9f-f009-4041-a38f-829a9a1f2819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:24.694662 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50ea9f-f009-4041-a38f-829a9a1f2819 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:24.695477 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc50ea9f-f009-4041-a38f-829a9a1f2819 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:24.697402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc50ea9f-f009-4041-a38f-829a9a1f2819 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:24.701037 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b193bcf3-5d62-4f5e-82e1-b1c4e02a61df admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:24.706312 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946700c4-8a74-478a-8829-71215a24c1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:24.707512 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946700c4-8a74-478a-8829-71215a24c1a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:24.707802 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946700c4-8a74-478a-8829-71215a24c1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:24.708086 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946700c4-8a74-478a-8829-71215a24c1a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:24.708797 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946700c4-8a74-478a-8829-71215a24c1a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:24.711299 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946700c4-8a74-478a-8829-71215a24c1a5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:24.730923 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 982/1935] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:15:24 2022] POST /identity/v3/users => generated 288 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:24.730923 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50ea9f-f009-4041-a38f-829a9a1f2819 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:24.730923 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cc50ea9f-f009-4041-a38f-829a9a1f2819 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:24.754884 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50ea9f-f009-4041-a38f-829a9a1f2819 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:24.766531 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 954/1936] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:15:24 2022] POST /identity/v3/projects => generated 367 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:24.767191 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbda0cd5-0f94-4b19-9a36-2f69d4e382dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:24.767552 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946700c4-8a74-478a-8829-71215a24c1a5 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:24.768913 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda0cd5-0f94-4b19-9a36-2f69d4e382dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:24.769583 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda0cd5-0f94-4b19-9a36-2f69d4e382dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:24.770170 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbda0cd5-0f94-4b19-9a36-2f69d4e382dd admin admin] PATH_INFO: `/v3/projects/031e27c85feb4caea7b57db4f7b10bb3/users/23a498528d784aca8d94124d45d750a8/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:24.788642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:24.798769 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbda0cd5-0f94-4b19-9a36-2f69d4e382dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=031e27c85feb4caea7b57db4f7b10bb3, user_id=23a498528d784aca8d94124d45d750a8, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:24.803523 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbda0cd5-0f94-4b19-9a36-2f69d4e382dd admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:24.818303 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 983/1937] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:24 2022] PUT /identity/v3/projects/031e27c85feb4caea7b57db4f7b10bb3/users/23a498528d784aca8d94124d45d750a8/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:24.819176 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbda0cd5-0f94-4b19-9a36-2f69d4e382dd admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:24.850999 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946700c4-8a74-478a-8829-71215a24c1a5 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:24.905088 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbda0cd5-0f94-4b19-9a36-2f69d4e382dd admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:24.910314 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6004c349-a7c8-4e54-b54c-b2031d0510f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:24.911111 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6004c349-a7c8-4e54-b54c-b2031d0510f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:24.911497 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6004c349-a7c8-4e54-b54c-b2031d0510f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:24.911865 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6004c349-a7c8-4e54-b54c-b2031d0510f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:24.912723 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6004c349-a7c8-4e54-b54c-b2031d0510f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:24.914896 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6004c349-a7c8-4e54-b54c-b2031d0510f5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:24.927534 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b6ee3-5d8d-4796-90e9-708a1f269bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:24.928172 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b6ee3-5d8d-4796-90e9-708a1f269bbc admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:24.928567 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b6ee3-5d8d-4796-90e9-708a1f269bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:24.928973 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b6ee3-5d8d-4796-90e9-708a1f269bbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:24.929940 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b6ee3-5d8d-4796-90e9-708a1f269bbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:24.932046 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b6ee3-5d8d-4796-90e9-708a1f269bbc admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:24.941067 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 984/1938] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:15:24 2022] POST /identity/v3/users => generated 284 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:24.941924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6004c349-a7c8-4e54-b54c-b2031d0510f5 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:24.947177 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 955/1939] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:24.948469 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b6ee3-5d8d-4796-90e9-708a1f269bbc admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:24.951275 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-331b6ee3-5d8d-4796-90e9-708a1f269bbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:24.953096 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6004c349-a7c8-4e54-b54c-b2031d0510f5 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:24.957967 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69964f84-4b51-46db-b467-c2afa21722aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:24.958701 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69964f84-4b51-46db-b467-c2afa21722aa admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:24.959089 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69964f84-4b51-46db-b467-c2afa21722aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:24.959557 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69964f84-4b51-46db-b467-c2afa21722aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:24.960473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69964f84-4b51-46db-b467-c2afa21722aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:24.962159 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69964f84-4b51-46db-b467-c2afa21722aa admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:24.965054 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b6ee3-5d8d-4796-90e9-708a1f269bbc admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:24.970441 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa17e6a-eda3-426d-8c69-bf90dab0f6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:24.971333 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa17e6a-eda3-426d-8c69-bf90dab0f6fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:24.971620 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa17e6a-eda3-426d-8c69-bf90dab0f6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:24.971899 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa17e6a-eda3-426d-8c69-bf90dab0f6fd admin admin] PATH_INFO: `/v3/projects/071100ed27e2483c9d0c7ca3b50f577e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:24.973181 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa17e6a-eda3-426d-8c69-bf90dab0f6fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=071100ed27e2483c9d0c7ca3b50f577e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:24.975123 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa17e6a-eda3-426d-8c69-bf90dab0f6fd admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:24.979179 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 985/1940] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:24.980876 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69964f84-4b51-46db-b467-c2afa21722aa admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:25.001447 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69964f84-4b51-46db-b467-c2afa21722aa admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:25.013612 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d2d63a-5252-4ed2-be46-f9cbf28d2a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:25.014536 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d2d63a-5252-4ed2-be46-f9cbf28d2a1a admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:25.014672 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d2d63a-5252-4ed2-be46-f9cbf28d2a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:25.014963 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d2d63a-5252-4ed2-be46-f9cbf28d2a1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:25.015704 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d2d63a-5252-4ed2-be46-f9cbf28d2a1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:25.017605 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d2d63a-5252-4ed2-be46-f9cbf28d2a1a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:25.036184 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5aa17e6a-eda3-426d-8c69-bf90dab0f6fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '071100ed27e2483c9d0c7ca3b50f577e', 'request_id': 'req-5aa17e6a-eda3-426d-8c69-bf90dab0f6fd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:25.037748 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:25.043377 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 986/1941] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:25.043377 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d2d63a-5252-4ed2-be46-f9cbf28d2a1a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:25.052685 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5aa17e6a-eda3-426d-8c69-bf90dab0f6fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '071100ed27e2483c9d0c7ca3b50f577e', 'request_id': 'req-5aa17e6a-eda3-426d-8c69-bf90dab0f6fd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:25.061329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d2d63a-5252-4ed2-be46-f9cbf28d2a1a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:25.065194 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5aa17e6a-eda3-426d-8c69-bf90dab0f6fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '071100ed27e2483c9d0c7ca3b50f577e', 'request_id': 'req-5aa17e6a-eda3-426d-8c69-bf90dab0f6fd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:25.081184 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5aa17e6a-eda3-426d-8c69-bf90dab0f6fd admin admin] The token cache is being invalidate because project 071100ed27e2483c9d0c7ca3b50f577e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:15:25.081653 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5aa17e6a-eda3-426d-8c69-bf90dab0f6fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:25.082497 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5aa17e6a-eda3-426d-8c69-bf90dab0f6fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:25.084130 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 956/1942] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:15:24 2022] DELETE /identity/v3/projects/071100ed27e2483c9d0c7ca3b50f577e => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:25.085203 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa17e6a-eda3-426d-8c69-bf90dab0f6fd admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:25.086652 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5aa17e6a-eda3-426d-8c69-bf90dab0f6fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:25.140427 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62927ad6-aac5-4121-a66a-4de990cb0d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:25.141018 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62927ad6-aac5-4121-a66a-4de990cb0d33 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:25.141313 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62927ad6-aac5-4121-a66a-4de990cb0d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:25.141595 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62927ad6-aac5-4121-a66a-4de990cb0d33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:25.142322 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62927ad6-aac5-4121-a66a-4de990cb0d33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:25.143956 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62927ad6-aac5-4121-a66a-4de990cb0d33 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:25.157659 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 987/1943] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:25.158751 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62927ad6-aac5-4121-a66a-4de990cb0d33 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:25.159947 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa17e6a-eda3-426d-8c69-bf90dab0f6fd admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:25.160463 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-62927ad6-aac5-4121-a66a-4de990cb0d33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:25.165042 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb72113-6a99-4b69-a2bd-8c1618b74998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:25.165930 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb72113-6a99-4b69-a2bd-8c1618b74998 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:25.166346 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb72113-6a99-4b69-a2bd-8c1618b74998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:25.166632 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb72113-6a99-4b69-a2bd-8c1618b74998 admin admin] PATH_INFO: `/v3/projects/031e27c85feb4caea7b57db4f7b10bb3/users/23a498528d784aca8d94124d45d750a8/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:25.169215 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb72113-6a99-4b69-a2bd-8c1618b74998 admin admin] RBAC: Authorizing `identity:create_grant(project_id=031e27c85feb4caea7b57db4f7b10bb3, user_id=23a498528d784aca8d94124d45d750a8, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:25.171992 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb72113-6a99-4b69-a2bd-8c1618b74998 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:25.174976 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62927ad6-aac5-4121-a66a-4de990cb0d33 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:25.180605 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffef9db-faaa-4239-9ef7-7c4b94a3dbda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:25.181282 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffef9db-faaa-4239-9ef7-7c4b94a3dbda admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:25.181552 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffef9db-faaa-4239-9ef7-7c4b94a3dbda admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:25.181828 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffef9db-faaa-4239-9ef7-7c4b94a3dbda admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:25.182666 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 957/1944] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:25 2022] PUT /identity/v3/projects/031e27c85feb4caea7b57db4f7b10bb3/users/23a498528d784aca8d94124d45d750a8/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:25.182783 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffef9db-faaa-4239-9ef7-7c4b94a3dbda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:25.183954 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b216e-6e40-401b-b62e-f853a824576e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:25.184268 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b216e-6e40-401b-b62e-f853a824576e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:25.184520 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1b216e-6e40-401b-b62e-f853a824576e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:25.184618 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffef9db-faaa-4239-9ef7-7c4b94a3dbda admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:25.196364 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 988/1945] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:25.197227 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffef9db-faaa-4239-9ef7-7c4b94a3dbda admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:25.199318 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-dffef9db-faaa-4239-9ef7-7c4b94a3dbda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:25.204658 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-9b1b216e-6e40-401b-b62e-f853a824576e None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:25.205889 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9b1b216e-6e40-401b-b62e-f853a824576e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:25.211626 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffef9db-faaa-4239-9ef7-7c4b94a3dbda admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:25.252729 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:25.268784 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 958/1946] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:25.270096 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc3afd4-06e7-4a9a-ac36-3585b288798a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:25.270395 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc3afd4-06e7-4a9a-ac36-3585b288798a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:25.270798 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc3afd4-06e7-4a9a-ac36-3585b288798a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:25.271373 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c75e6ee-2803-4ae0-a212-3d11c2597354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:25.273174 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75e6ee-2803-4ae0-a212-3d11c2597354 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:25.273622 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75e6ee-2803-4ae0-a212-3d11c2597354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:25.273901 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75e6ee-2803-4ae0-a212-3d11c2597354 admin admin] PATH_INFO: `/v3/projects/c3c4757ffbbb4f5ba3cda979c84a20df/users/f00d5ae33cb642979533f6cd5c27a02d/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:25.293105 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c75e6ee-2803-4ae0-a212-3d11c2597354 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3c4757ffbbb4f5ba3cda979c84a20df, user_id=f00d5ae33cb642979533f6cd5c27a02d, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:25.297643 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c75e6ee-2803-4ae0-a212-3d11c2597354 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:25.308221 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-fdc3afd4-06e7-4a9a-ac36-3585b288798a None None] MFA Rules not processed for user `23a498528d784aca8d94124d45d750a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:25.309540 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fdc3afd4-06e7-4a9a-ac36-3585b288798a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:25.313042 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 989/1947] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:25 2022] PUT /identity/v3/projects/c3c4757ffbbb4f5ba3cda979c84a20df/users/f00d5ae33cb642979533f6cd5c27a02d/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:25.314158 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c75e6ee-2803-4ae0-a212-3d11c2597354 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:25.383666 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:25.389927 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 959/1948] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:25 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:25.391541 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a8f6a-59de-4431-8398-6c58b3da9b02 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:25.392563 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a8f6a-59de-4431-8398-6c58b3da9b02 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:25.393211 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a8f6a-59de-4431-8398-6c58b3da9b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:25.394889 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c75e6ee-2803-4ae0-a212-3d11c2597354 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:25.400082 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d2ba4c-8b2d-4f7f-ab28-8d7a72e9083b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:25.400997 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2ba4c-8b2d-4f7f-ab28-8d7a72e9083b admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:25.401401 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2ba4c-8b2d-4f7f-ab28-8d7a72e9083b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:25.401768 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d2ba4c-8b2d-4f7f-ab28-8d7a72e9083b admin admin] PATH_INFO: `/v3/projects/30a492c71c0c4afd927bc0ad8c4c46fc/users/bc060e442609458d8c27c404d598906a/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:25.426235 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ee3a8f6a-59de-4431-8398-6c58b3da9b02 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:25.426777 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ee3a8f6a-59de-4431-8398-6c58b3da9b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:25.427030 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d2ba4c-8b2d-4f7f-ab28-8d7a72e9083b admin admin] RBAC: Authorizing `identity:create_grant(project_id=30a492c71c0c4afd927bc0ad8c4c46fc, user_id=bc060e442609458d8c27c404d598906a, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:25.433412 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d2ba4c-8b2d-4f7f-ab28-8d7a72e9083b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:25.437065 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 960/1949] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:25.438554 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c463c1b-6416-4741-8f8b-403e2d3baeb5 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:25.438699 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c463c1b-6416-4741-8f8b-403e2d3baeb5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:25.438956 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c463c1b-6416-4741-8f8b-403e2d3baeb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:25.466284 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 990/1950] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:25 2022] PUT /identity/v3/projects/30a492c71c0c4afd927bc0ad8c4c46fc/users/bc060e442609458d8c27c404d598906a/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:25.467607 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d2ba4c-8b2d-4f7f-ab28-8d7a72e9083b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:25.470161 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b9d2ba4c-8b2d-4f7f-ab28-8d7a72e9083b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:25.485628 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5c463c1b-6416-4741-8f8b-403e2d3baeb5 None None] MFA Rules not processed for user `f00d5ae33cb642979533f6cd5c27a02d`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:25.486933 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5c463c1b-6416-4741-8f8b-403e2d3baeb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:25.543207 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d2ba4c-8b2d-4f7f-ab28-8d7a72e9083b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:25.551005 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba56c07-6109-421c-a31c-e84af121ff00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:25.551926 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba56c07-6109-421c-a31c-e84af121ff00 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:25.552434 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba56c07-6109-421c-a31c-e84af121ff00 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:25.552853 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba56c07-6109-421c-a31c-e84af121ff00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:25.572327 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 961/1951] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:25.573269 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c463c1b-6416-4741-8f8b-403e2d3baeb5 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:25.628373 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba56c07-6109-421c-a31c-e84af121ff00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:25.635150 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba56c07-6109-421c-a31c-e84af121ff00 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:25.640911 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:25.648804 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 991/1952] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:25 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:25.650594 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be19411b-c9b1-485d-a08d-6adf4e26273d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:25.652068 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be19411b-c9b1-485d-a08d-6adf4e26273d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:25.652920 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be19411b-c9b1-485d-a08d-6adf4e26273d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:25.659747 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c463c1b-6416-4741-8f8b-403e2d3baeb5 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:25.666257 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d251ecd-432e-4545-8fc9-ce53f5c33d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:25.667383 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d251ecd-432e-4545-8fc9-ce53f5c33d0c admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:25.668389 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d251ecd-432e-4545-8fc9-ce53f5c33d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:25.668389 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d251ecd-432e-4545-8fc9-ce53f5c33d0c admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:25.669077 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d251ecd-432e-4545-8fc9-ce53f5c33d0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:15:25.669371 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d251ecd-432e-4545-8fc9-ce53f5c33d0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:25.671082 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d251ecd-432e-4545-8fc9-ce53f5c33d0c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:25.687532 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 962/1953] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:15:25.688554 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d251ecd-432e-4545-8fc9-ce53f5c33d0c admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:25.689303 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-be19411b-c9b1-485d-a08d-6adf4e26273d None None] MFA Rules not processed for user `bc060e442609458d8c27c404d598906a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:25.690883 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-be19411b-c9b1-485d-a08d-6adf4e26273d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:25.706389 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d251ecd-432e-4545-8fc9-ce53f5c33d0c admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:25.719922 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abdfe1b-9ef3-4235-a597-da620006d203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:25.720760 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abdfe1b-9ef3-4235-a597-da620006d203 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:25.721055 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abdfe1b-9ef3-4235-a597-da620006d203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:25.721339 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abdfe1b-9ef3-4235-a597-da620006d203 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:25.722043 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abdfe1b-9ef3-4235-a597-da620006d203 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:25.724459 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abdfe1b-9ef3-4235-a597-da620006d203 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:25.766776 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 963/1954] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:15:25 2022] POST /identity/v3/projects => generated 373 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:25.767943 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abdfe1b-9ef3-4235-a597-da620006d203 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:25.797212 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 992/1955] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:25 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:25.799500 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be19411b-c9b1-485d-a08d-6adf4e26273d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:25.832026 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be19411b-c9b1-485d-a08d-6adf4e26273d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:25.861401 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:25.875130 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abdfe1b-9ef3-4235-a597-da620006d203 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:25.882078 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dd23b5-a9f8-406e-9dd1-64340ef483f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:25.882078 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd23b5-a9f8-406e-9dd1-64340ef483f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:25.882078 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd23b5-a9f8-406e-9dd1-64340ef483f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:25.882078 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd23b5-a9f8-406e-9dd1-64340ef483f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:25.883163 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dd23b5-a9f8-406e-9dd1-64340ef483f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:25.885262 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dd23b5-a9f8-406e-9dd1-64340ef483f1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:25.922086 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcb0a7b-6cdc-46b0-87e9-6288b892e5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:25.923372 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb0a7b-6cdc-46b0-87e9-6288b892e5c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:25.923971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb0a7b-6cdc-46b0-87e9-6288b892e5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:25.928211 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 964/1956] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:15:25 2022] POST /identity/v3/projects => generated 375 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:25.928211 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dd23b5-a9f8-406e-9dd1-64340ef483f1 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:25.929058 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcb0a7b-6cdc-46b0-87e9-6288b892e5c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:25.930043 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcb0a7b-6cdc-46b0-87e9-6288b892e5c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:25.933721 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcb0a7b-6cdc-46b0-87e9-6288b892e5c0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:25.971092 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 993/1957] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:15:25 2022] POST /identity/v3/users => generated 293 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:25.972425 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcb0a7b-6cdc-46b0-87e9-6288b892e5c0 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:25.990015 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcb0a7b-6cdc-46b0-87e9-6288b892e5c0 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:26.008065 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dd23b5-a9f8-406e-9dd1-64340ef483f1 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:26.014256 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a18bf7-dbe7-40e1-aff5-782ebfa40f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:26.015176 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a18bf7-dbe7-40e1-aff5-782ebfa40f5c admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:26.015466 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a18bf7-dbe7-40e1-aff5-782ebfa40f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:26.015748 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a18bf7-dbe7-40e1-aff5-782ebfa40f5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:26.016616 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a18bf7-dbe7-40e1-aff5-782ebfa40f5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:26.019138 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a18bf7-dbe7-40e1-aff5-782ebfa40f5c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:26.057551 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 965/1958] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:15:25 2022] POST /identity/v3/projects => generated 367 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:26.059036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a18bf7-dbe7-40e1-aff5-782ebfa40f5c admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:26.084936 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c398dc94-481e-4337-a474-84b64c2f11df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:26.086767 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c398dc94-481e-4337-a474-84b64c2f11df admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:26.087106 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c398dc94-481e-4337-a474-84b64c2f11df admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:26.087422 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c398dc94-481e-4337-a474-84b64c2f11df admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:26.092068 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c398dc94-481e-4337-a474-84b64c2f11df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:26.097394 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c398dc94-481e-4337-a474-84b64c2f11df admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:26.140955 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 994/1959] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:15:25 2022] POST /identity/v3/users => generated 288 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:26.143554 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c398dc94-481e-4337-a474-84b64c2f11df admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:26.169504 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c398dc94-481e-4337-a474-84b64c2f11df admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:26.191589 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30136b14-d46b-4135-aa93-d8ea8dabfe73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:26.192365 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30136b14-d46b-4135-aa93-d8ea8dabfe73 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:26.193933 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30136b14-d46b-4135-aa93-d8ea8dabfe73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:26.194994 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30136b14-d46b-4135-aa93-d8ea8dabfe73 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:26.197720 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30136b14-d46b-4135-aa93-d8ea8dabfe73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:26.202333 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30136b14-d46b-4135-aa93-d8ea8dabfe73 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:26.203927 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a18bf7-dbe7-40e1-aff5-782ebfa40f5c admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:26.209974 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:26.212951 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4510b1d-ed69-4d98-baaa-ccf36ce174a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:26.214456 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4510b1d-ed69-4d98-baaa-ccf36ce174a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:26.215483 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4510b1d-ed69-4d98-baaa-ccf36ce174a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:26.215836 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4510b1d-ed69-4d98-baaa-ccf36ce174a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:26.216839 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4510b1d-ed69-4d98-baaa-ccf36ce174a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:26.218808 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4510b1d-ed69-4d98-baaa-ccf36ce174a8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:26.235066 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 995/1960] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:15:26 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:26.235066 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30136b14-d46b-4135-aa93-d8ea8dabfe73 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:26.235440 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 966/1961] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:26.241059 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4510b1d-ed69-4d98-baaa-ccf36ce174a8 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:26.251056 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30136b14-d46b-4135-aa93-d8ea8dabfe73 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:26.259965 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4510b1d-ed69-4d98-baaa-ccf36ce174a8 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:26.262877 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8663d0f-c761-4057-8009-f576f63dd744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:26.263832 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8663d0f-c761-4057-8009-f576f63dd744 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:26.264283 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8663d0f-c761-4057-8009-f576f63dd744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:26.264805 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8663d0f-c761-4057-8009-f576f63dd744 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:26.265691 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d15b611-5abe-4739-8e95-29b0574fd028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:26.266351 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d15b611-5abe-4739-8e95-29b0574fd028 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:26.266636 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d15b611-5abe-4739-8e95-29b0574fd028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:26.266944 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d15b611-5abe-4739-8e95-29b0574fd028 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:26.267668 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d15b611-5abe-4739-8e95-29b0574fd028 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:26.269273 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d15b611-5abe-4739-8e95-29b0574fd028 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:26.269391 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8663d0f-c761-4057-8009-f576f63dd744 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:26.275848 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8663d0f-c761-4057-8009-f576f63dd744 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:26.283855 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 967/1962] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:26.287269 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d15b611-5abe-4739-8e95-29b0574fd028 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:26.299843 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d15b611-5abe-4739-8e95-29b0574fd028 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:26.309710 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 996/1963] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:26.309710 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8663d0f-c761-4057-8009-f576f63dd744 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:26.311690 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f777daf5-2c46-432c-8811-4e03100f1d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:26.312594 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777daf5-2c46-432c-8811-4e03100f1d1f admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:26.313398 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777daf5-2c46-432c-8811-4e03100f1d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:26.313563 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777daf5-2c46-432c-8811-4e03100f1d1f admin admin] PATH_INFO: `/v3/projects/78c337fa58c04e3db006478434dbc624/users/a2a6874387fd460984cb8e84856f511f/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:26.326840 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8663d0f-c761-4057-8009-f576f63dd744 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:26.335479 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779bfa7b-ffdd-45cf-8acd-1872772befd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:26.336981 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779bfa7b-ffdd-45cf-8acd-1872772befd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:26.338136 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f777daf5-2c46-432c-8811-4e03100f1d1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=78c337fa58c04e3db006478434dbc624, user_id=a2a6874387fd460984cb8e84856f511f, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:26.338498 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779bfa7b-ffdd-45cf-8acd-1872772befd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:26.339150 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779bfa7b-ffdd-45cf-8acd-1872772befd0 admin admin] PATH_INFO: `/v3/projects/ec8c19b199db4ae3aeca0c2aeb030fd5/users/00b72cc486eb411193e3dfba88f192d2/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:26.343557 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f777daf5-2c46-432c-8811-4e03100f1d1f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:26.365683 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779bfa7b-ffdd-45cf-8acd-1872772befd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec8c19b199db4ae3aeca0c2aeb030fd5, user_id=00b72cc486eb411193e3dfba88f192d2, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:26.368891 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779bfa7b-ffdd-45cf-8acd-1872772befd0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:26.371075 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 968/1964] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:26 2022] PUT /identity/v3/projects/78c337fa58c04e3db006478434dbc624/users/a2a6874387fd460984cb8e84856f511f/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:26.373478 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f777daf5-2c46-432c-8811-4e03100f1d1f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:26.386181 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 997/1965] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:26 2022] PUT /identity/v3/projects/ec8c19b199db4ae3aeca0c2aeb030fd5/users/00b72cc486eb411193e3dfba88f192d2/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:26.386181 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779bfa7b-ffdd-45cf-8acd-1872772befd0 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:26.405924 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:26.451810 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779bfa7b-ffdd-45cf-8acd-1872772befd0 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:26.452327 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f777daf5-2c46-432c-8811-4e03100f1d1f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:26.457206 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a49d53a-ca38-468c-bc0f-5244de63f9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:26.457356 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8a057-62a3-4b71-84db-98f14c2ed671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:26.457877 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a49d53a-ca38-468c-bc0f-5244de63f9c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:26.458075 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8a057-62a3-4b71-84db-98f14c2ed671 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:26.458174 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a49d53a-ca38-468c-bc0f-5244de63f9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:26.458428 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a49d53a-ca38-468c-bc0f-5244de63f9c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:26.458670 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8a057-62a3-4b71-84db-98f14c2ed671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:26.459047 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8a057-62a3-4b71-84db-98f14c2ed671 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:26.459188 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a49d53a-ca38-468c-bc0f-5244de63f9c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:26.460034 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a8a057-62a3-4b71-84db-98f14c2ed671 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:26.460787 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a49d53a-ca38-468c-bc0f-5244de63f9c6 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:26.461760 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a8a057-62a3-4b71-84db-98f14c2ed671 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:26.473298 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 969/1966] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:26.474189 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a49d53a-ca38-468c-bc0f-5244de63f9c6 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:26.480786 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 998/1967] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:26.484549 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8a057-62a3-4b71-84db-98f14c2ed671 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:26.486366 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a49d53a-ca38-468c-bc0f-5244de63f9c6 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:26.493248 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad08a4c-da10-4507-8112-61189d48fcb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:26.493764 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad08a4c-da10-4507-8112-61189d48fcb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:26.494054 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad08a4c-da10-4507-8112-61189d48fcb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:26.494335 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad08a4c-da10-4507-8112-61189d48fcb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:26.495127 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad08a4c-da10-4507-8112-61189d48fcb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:26.496769 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad08a4c-da10-4507-8112-61189d48fcb4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:26.498354 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8a057-62a3-4b71-84db-98f14c2ed671 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:26.506812 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78953050-71fe-4317-8ea3-0e1e94905084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:26.507411 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 970/1968] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:26.507748 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78953050-71fe-4317-8ea3-0e1e94905084 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:26.508033 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78953050-71fe-4317-8ea3-0e1e94905084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:26.508307 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78953050-71fe-4317-8ea3-0e1e94905084 admin admin] PATH_INFO: `/v3/projects/5807ebd6499d40cf8461813e1006e7d6/users/7e3961e4d389465d9a3191408cbaaa5b/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:26.508391 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad08a4c-da10-4507-8112-61189d48fcb4 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:26.519423 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad08a4c-da10-4507-8112-61189d48fcb4 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:26.523615 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78953050-71fe-4317-8ea3-0e1e94905084 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5807ebd6499d40cf8461813e1006e7d6, user_id=7e3961e4d389465d9a3191408cbaaa5b, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:26.524173 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8da0fd-873d-4d9a-8dac-25e2dacc1db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:26.525035 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8da0fd-873d-4d9a-8dac-25e2dacc1db9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:26.525321 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8da0fd-873d-4d9a-8dac-25e2dacc1db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:26.525600 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8da0fd-873d-4d9a-8dac-25e2dacc1db9 admin admin] PATH_INFO: `/v3/projects/78c337fa58c04e3db006478434dbc624/users/a2a6874387fd460984cb8e84856f511f/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:26.526841 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78953050-71fe-4317-8ea3-0e1e94905084 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:26.528478 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8da0fd-873d-4d9a-8dac-25e2dacc1db9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78c337fa58c04e3db006478434dbc624, user_id=a2a6874387fd460984cb8e84856f511f, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:26.531164 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8da0fd-873d-4d9a-8dac-25e2dacc1db9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:26.540177 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 999/1969] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:26 2022] PUT /identity/v3/projects/5807ebd6499d40cf8461813e1006e7d6/users/7e3961e4d389465d9a3191408cbaaa5b/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:26.541089 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78953050-71fe-4317-8ea3-0e1e94905084 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:26.543926 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 971/1970] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:26 2022] PUT /identity/v3/projects/78c337fa58c04e3db006478434dbc624/users/a2a6874387fd460984cb8e84856f511f/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:26.553278 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8da0fd-873d-4d9a-8dac-25e2dacc1db9 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:26.608107 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78953050-71fe-4317-8ea3-0e1e94905084 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:26.622460 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8da0fd-873d-4d9a-8dac-25e2dacc1db9 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:26.627119 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67118c21-056d-42a9-9060-f0821b7f5ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:26.628001 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67118c21-056d-42a9-9060-f0821b7f5ea3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:26.628283 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67118c21-056d-42a9-9060-f0821b7f5ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:26.628559 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67118c21-056d-42a9-9060-f0821b7f5ea3 admin admin] PATH_INFO: `/v3/projects/ec8c19b199db4ae3aeca0c2aeb030fd5/users/00b72cc486eb411193e3dfba88f192d2/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:26.630515 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67118c21-056d-42a9-9060-f0821b7f5ea3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec8c19b199db4ae3aeca0c2aeb030fd5, user_id=00b72cc486eb411193e3dfba88f192d2, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:26.633203 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67118c21-056d-42a9-9060-f0821b7f5ea3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:26.643210 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:26.655145 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 972/1971] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:26 2022] PUT /identity/v3/projects/ec8c19b199db4ae3aeca0c2aeb030fd5/users/00b72cc486eb411193e3dfba88f192d2/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:26.655145 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb3a16f-fc83-4ed4-bd38-2a43ef979a66 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:26.655145 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb3a16f-fc83-4ed4-bd38-2a43ef979a66 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:26.655145 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb3a16f-fc83-4ed4-bd38-2a43ef979a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:26.673031 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4316b3-1f6a-4bec-b203-266a96601b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:26.674306 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4316b3-1f6a-4bec-b203-266a96601b78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:26.674559 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4316b3-1f6a-4bec-b203-266a96601b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:26.674900 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4316b3-1f6a-4bec-b203-266a96601b78 admin admin] PATH_INFO: `/v3/users/a50c6cab092b47329c9f5802168ca355` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:26.676178 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4316b3-1f6a-4bec-b203-266a96601b78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a50c6cab092b47329c9f5802168ca355)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:26.678797 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4316b3-1f6a-4bec-b203-266a96601b78 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:26.685938 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-1fb3a16f-fc83-4ed4-bd38-2a43ef979a66 None None] MFA Rules not processed for user `7e3961e4d389465d9a3191408cbaaa5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:26.687188 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1fb3a16f-fc83-4ed4-bd38-2a43ef979a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:26.728760 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ab4316b3-1f6a-4bec-b203-266a96601b78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a50c6cab092b47329c9f5802168ca355'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:26.737779 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ab4316b3-1f6a-4bec-b203-266a96601b78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a50c6cab092b47329c9f5802168ca355'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:26.738615 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ab4316b3-1f6a-4bec-b203-266a96601b78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a50c6cab092b47329c9f5802168ca355'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:26.748536 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ab4316b3-1f6a-4bec-b203-266a96601b78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a50c6cab092b47329c9f5802168ca355'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:26.749407 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ab4316b3-1f6a-4bec-b203-266a96601b78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a50c6cab092b47329c9f5802168ca355'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:26.761597 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 973/1972] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:26.765814 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5aea77-2f93-4e9b-895a-bd8860a4da79 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:26.766679 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5aea77-2f93-4e9b-895a-bd8860a4da79 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:26.766679 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5aea77-2f93-4e9b-895a-bd8860a4da79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:26.780494 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1000/1973] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:15:26 2022] DELETE /identity/v3/users/a50c6cab092b47329c9f5802168ca355 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:26.780494 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18d307-65be-4444-b7d6-4629f653b245 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:26.780494 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18d307-65be-4444-b7d6-4629f653b245 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:26.780494 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a18d307-65be-4444-b7d6-4629f653b245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:26.800818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5b5aea77-2f93-4e9b-895a-bd8860a4da79 None None] MFA Rules not processed for user `a2a6874387fd460984cb8e84856f511f`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:26.802584 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5b5aea77-2f93-4e9b-895a-bd8860a4da79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:26.811942 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8a18d307-65be-4444-b7d6-4629f653b245 None None] MFA Rules not processed for user `00b72cc486eb411193e3dfba88f192d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:26.813457 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8a18d307-65be-4444-b7d6-4629f653b245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:26.815796 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:26.879871 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 974/1974] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:26 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:26.881094 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5aea77-2f93-4e9b-895a-bd8860a4da79 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:26.882356 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5b5aea77-2f93-4e9b-895a-bd8860a4da79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:26.908890 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1001/1975] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:26 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:26.910194 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e630f8df-6eff-49c0-9051-91c95278cadd None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:26.910489 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e630f8df-6eff-49c0-9051-91c95278cadd None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:26.910769 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e630f8df-6eff-49c0-9051-91c95278cadd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:26.929597 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e630f8df-6eff-49c0-9051-91c95278cadd None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:26.930921 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e630f8df-6eff-49c0-9051-91c95278cadd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:26.954701 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5aea77-2f93-4e9b-895a-bd8860a4da79 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:26.959528 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1dc386-c097-4360-bd39-c6086b3f3ce9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1afda9d6b1d64e6ab400b26124cf1290', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:26.960690 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1dc386-c097-4360-bd39-c6086b3f3ce9 service glance] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:26.960985 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1dc386-c097-4360-bd39-c6086b3f3ce9 service glance] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:26.961894 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1dc386-c097-4360-bd39-c6086b3f3ce9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:26.963432 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bf1dc386-c097-4360-bd39-c6086b3f3ce9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:26.968482 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:26.999571 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1002/1976] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:27.000424 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e630f8df-6eff-49c0-9051-91c95278cadd None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:27.001896 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e630f8df-6eff-49c0-9051-91c95278cadd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:27.038082 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1dc386-c097-4360-bd39-c6086b3f3ce9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:27.039604 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1dc386-c097-4360-bd39-c6086b3f3ce9 service glance] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:27.049523 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 975/1977] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:27.050410 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1dc386-c097-4360-bd39-c6086b3f3ce9 service glance] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:27.051748 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bf1dc386-c097-4360-bd39-c6086b3f3ce9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:27.074425 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e630f8df-6eff-49c0-9051-91c95278cadd None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:27.079642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3abb68e-06d8-4311-b293-627670d8a9fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:27.080563 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3abb68e-06d8-4311-b293-627670d8a9fb service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:27.080921 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3abb68e-06d8-4311-b293-627670d8a9fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:27.081345 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3abb68e-06d8-4311-b293-627670d8a9fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:27.085214 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c3abb68e-06d8-4311-b293-627670d8a9fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:27.119419 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1dc386-c097-4360-bd39-c6086b3f3ce9 service glance] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:27.124311 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05f9ee3-54de-44db-a20b-dce37e2ec806 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:27.125076 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f9ee3-54de-44db-a20b-dce37e2ec806 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:27.125485 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f9ee3-54de-44db-a20b-dce37e2ec806 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:27.125831 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f9ee3-54de-44db-a20b-dce37e2ec806 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:27.139271 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05f9ee3-54de-44db-a20b-dce37e2ec806 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:27.141223 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05f9ee3-54de-44db-a20b-dce37e2ec806 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:27.153275 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 976/1978] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:27 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:27.154812 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05f9ee3-54de-44db-a20b-dce37e2ec806 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:27.156251 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e05f9ee3-54de-44db-a20b-dce37e2ec806 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:27.163618 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3abb68e-06d8-4311-b293-627670d8a9fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:27.165138 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:27.166109 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3abb68e-06d8-4311-b293-627670d8a9fb service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:27.186329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1003/1979] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:26 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:27.239932 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05f9ee3-54de-44db-a20b-dce37e2ec806 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:27.244746 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b39c2d-103d-4379-9bb1-871f11cabd3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:27.245535 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b39c2d-103d-4379-9bb1-871f11cabd3a service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:27.246018 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b39c2d-103d-4379-9bb1-871f11cabd3a service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:27.246350 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b39c2d-103d-4379-9bb1-871f11cabd3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:27.263545 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b39c2d-103d-4379-9bb1-871f11cabd3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:27.265591 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b39c2d-103d-4379-9bb1-871f11cabd3a service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:27.281359 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 977/1980] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:27.626066 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3abb68e-06d8-4311-b293-627670d8a9fb service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:27.640241 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3abb68e-06d8-4311-b293-627670d8a9fb service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:27.644395 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624dfef-169a-4773-a125-96fe5831fe17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:27.645104 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624dfef-169a-4773-a125-96fe5831fe17 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:27.645397 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624dfef-169a-4773-a125-96fe5831fe17 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:27.645724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624dfef-169a-4773-a125-96fe5831fe17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:27.649513 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b39c2d-103d-4379-9bb1-871f11cabd3a service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:27.650118 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2624dfef-169a-4773-a125-96fe5831fe17 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:27.651223 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-71b39c2d-103d-4379-9bb1-871f11cabd3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:27.673477 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b39c2d-103d-4379-9bb1-871f11cabd3a service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:27.684315 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9211e0bf-a1d8-4585-9bd1-8687e951e609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:27.685390 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9211e0bf-a1d8-4585-9bd1-8687e951e609 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:27.685744 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9211e0bf-a1d8-4585-9bd1-8687e951e609 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:27.686046 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9211e0bf-a1d8-4585-9bd1-8687e951e609 admin admin] PATH_INFO: `/v3/projects/bc5be122b10744a09a7397129532ea26` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:27.689140 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9211e0bf-a1d8-4585-9bd1-8687e951e609 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc5be122b10744a09a7397129532ea26)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:27.692499 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9211e0bf-a1d8-4585-9bd1-8687e951e609 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:27.758536 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2624dfef-169a-4773-a125-96fe5831fe17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:27.760893 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2624dfef-169a-4773-a125-96fe5831fe17 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:27.772043 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9211e0bf-a1d8-4585-9bd1-8687e951e609 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc5be122b10744a09a7397129532ea26', 'request_id': 'req-9211e0bf-a1d8-4585-9bd1-8687e951e609'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:27.775050 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1004/1981] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:27 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:27.779308 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9211e0bf-a1d8-4585-9bd1-8687e951e609 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc5be122b10744a09a7397129532ea26', 'request_id': 'req-9211e0bf-a1d8-4585-9bd1-8687e951e609'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:27.788203 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9211e0bf-a1d8-4585-9bd1-8687e951e609 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc5be122b10744a09a7397129532ea26', 'request_id': 'req-9211e0bf-a1d8-4585-9bd1-8687e951e609'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:27.829016 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9211e0bf-a1d8-4585-9bd1-8687e951e609 admin admin] The token cache is being invalidate because project bc5be122b10744a09a7397129532ea26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:15:27.831292 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9211e0bf-a1d8-4585-9bd1-8687e951e609 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:27.835160 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9211e0bf-a1d8-4585-9bd1-8687e951e609 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:27.839352 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 978/1982] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:15:27 2022] DELETE /identity/v3/projects/bc5be122b10744a09a7397129532ea26 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:15:27.844166 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624dfef-169a-4773-a125-96fe5831fe17 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:27.866266 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624dfef-169a-4773-a125-96fe5831fe17 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:27.963212 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:28.023814 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909a4b3a-f1ce-4a5f-89a5-ebeb9ae4bff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:28.025880 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a4b3a-f1ce-4a5f-89a5-ebeb9ae4bff2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:28.032780 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a4b3a-f1ce-4a5f-89a5-ebeb9ae4bff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:28.033427 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a4b3a-f1ce-4a5f-89a5-ebeb9ae4bff2 admin admin] PATH_INFO: `/v3/users/23a498528d784aca8d94124d45d750a8` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:28.036041 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909a4b3a-f1ce-4a5f-89a5-ebeb9ae4bff2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23a498528d784aca8d94124d45d750a8)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:28.038263 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909a4b3a-f1ce-4a5f-89a5-ebeb9ae4bff2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:28.113455 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-909a4b3a-f1ce-4a5f-89a5-ebeb9ae4bff2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23a498528d784aca8d94124d45d750a8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:28.151816 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-909a4b3a-f1ce-4a5f-89a5-ebeb9ae4bff2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23a498528d784aca8d94124d45d750a8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:28.155250 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-909a4b3a-f1ce-4a5f-89a5-ebeb9ae4bff2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23a498528d784aca8d94124d45d750a8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:28.169574 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-909a4b3a-f1ce-4a5f-89a5-ebeb9ae4bff2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23a498528d784aca8d94124d45d750a8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:28.170527 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-909a4b3a-f1ce-4a5f-89a5-ebeb9ae4bff2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23a498528d784aca8d94124d45d750a8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:28.193823 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1005/1983] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:15:27 2022] DELETE /identity/v3/users/23a498528d784aca8d94124d45d750a8 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:28.619591 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9211e0bf-a1d8-4585-9bd1-8687e951e609 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:28.621920 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9211e0bf-a1d8-4585-9bd1-8687e951e609 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:28.671707 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9211e0bf-a1d8-4585-9bd1-8687e951e609 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:28.746094 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966ac90-b218-4a64-8113-87d297924460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:28.747013 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966ac90-b218-4a64-8113-87d297924460 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:28.747301 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966ac90-b218-4a64-8113-87d297924460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:28.747579 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966ac90-b218-4a64-8113-87d297924460 admin admin] PATH_INFO: `/v3/projects/031e27c85feb4caea7b57db4f7b10bb3` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:28.748777 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9966ac90-b218-4a64-8113-87d297924460 admin admin] RBAC: Authorizing `identity:delete_project(project_id=031e27c85feb4caea7b57db4f7b10bb3)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:28.750701 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9966ac90-b218-4a64-8113-87d297924460 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:28.792780 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:28.809317 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9966ac90-b218-4a64-8113-87d297924460 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '031e27c85feb4caea7b57db4f7b10bb3', 'request_id': 'req-9966ac90-b218-4a64-8113-87d297924460'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:28.815792 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9966ac90-b218-4a64-8113-87d297924460 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '031e27c85feb4caea7b57db4f7b10bb3', 'request_id': 'req-9966ac90-b218-4a64-8113-87d297924460'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:28.823018 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9966ac90-b218-4a64-8113-87d297924460 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '031e27c85feb4caea7b57db4f7b10bb3', 'request_id': 'req-9966ac90-b218-4a64-8113-87d297924460'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:28.837716 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9966ac90-b218-4a64-8113-87d297924460 admin admin] The token cache is being invalidate because project 031e27c85feb4caea7b57db4f7b10bb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:15:28.838007 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9966ac90-b218-4a64-8113-87d297924460 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:28.838661 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9966ac90-b218-4a64-8113-87d297924460 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:28.840327 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 979/1984] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:15:28 2022] DELETE /identity/v3/projects/031e27c85feb4caea7b57db4f7b10bb3 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:28.845687 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7475e-721f-4260-900c-5a46251976c3 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:28.846267 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7475e-721f-4260-900c-5a46251976c3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:28.846466 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7475e-721f-4260-900c-5a46251976c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:28.865707 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6ad7475e-721f-4260-900c-5a46251976c3 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:28.866950 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6ad7475e-721f-4260-900c-5a46251976c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:28.937733 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1006/1985] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:28.949648 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947645f9-8a43-43fe-b4c9-9c1dd3c14676 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:28.950135 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947645f9-8a43-43fe-b4c9-9c1dd3c14676 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:28.950561 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947645f9-8a43-43fe-b4c9-9c1dd3c14676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:28.970050 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-947645f9-8a43-43fe-b4c9-9c1dd3c14676 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:28.971324 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-947645f9-8a43-43fe-b4c9-9c1dd3c14676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:28.981262 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 980/1986] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:28.985483 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7475e-721f-4260-900c-5a46251976c3 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:28.996126 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7475e-721f-4260-900c-5a46251976c3 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:29.001175 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3fda1-a90f-4168-95b9-4adb4b4b9c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:29.001912 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3fda1-a90f-4168-95b9-4adb4b4b9c7f admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:29.002304 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3fda1-a90f-4168-95b9-4adb4b4b9c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:29.002649 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a3fda1-a90f-4168-95b9-4adb4b4b9c7f admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:29.003525 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a3fda1-a90f-4168-95b9-4adb4b4b9c7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:15:29.003895 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a3fda1-a90f-4168-95b9-4adb4b4b9c7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:29.005521 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a3fda1-a90f-4168-95b9-4adb4b4b9c7f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:29.023719 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1007/1987] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:15:29.024918 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947645f9-8a43-43fe-b4c9-9c1dd3c14676 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:29.038645 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947645f9-8a43-43fe-b4c9-9c1dd3c14676 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:29.044883 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f302dab-6e76-4fc6-902d-9e78bb3a56a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:29.045595 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f302dab-6e76-4fc6-902d-9e78bb3a56a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:29.045901 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f302dab-6e76-4fc6-902d-9e78bb3a56a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:29.046205 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f302dab-6e76-4fc6-902d-9e78bb3a56a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:29.046992 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f302dab-6e76-4fc6-902d-9e78bb3a56a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:29.049175 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f302dab-6e76-4fc6-902d-9e78bb3a56a1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:29.089463 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 981/1988] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:15:29 2022] POST /identity/v3/projects => generated 361 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:29.090572 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3fda1-a90f-4168-95b9-4adb4b4b9c7f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:29.116974 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a3fda1-a90f-4168-95b9-4adb4b4b9c7f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:29.212279 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:29.257324 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a46224-cb5a-4f6f-89ec-2a2011bbeef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:29.258660 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a46224-cb5a-4f6f-89ec-2a2011bbeef6 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:29.259579 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a46224-cb5a-4f6f-89ec-2a2011bbeef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:29.260141 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a46224-cb5a-4f6f-89ec-2a2011bbeef6 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:29.263738 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a46224-cb5a-4f6f-89ec-2a2011bbeef6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:29.265754 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a46224-cb5a-4f6f-89ec-2a2011bbeef6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:29.299572 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1008/1989] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:15:29 2022] POST /identity/v3/users => generated 281 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:29.305452 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f302dab-6e76-4fc6-902d-9e78bb3a56a1 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:29.328184 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f302dab-6e76-4fc6-902d-9e78bb3a56a1 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:29.347435 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ea21d-0bdf-4655-b158-72454a3ee69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:29.348511 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ea21d-0bdf-4655-b158-72454a3ee69f admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:29.349003 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ea21d-0bdf-4655-b158-72454a3ee69f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:29.349625 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ea21d-0bdf-4655-b158-72454a3ee69f admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:29.350591 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5ea21d-0bdf-4655-b158-72454a3ee69f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:29.352461 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5ea21d-0bdf-4655-b158-72454a3ee69f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:29.375859 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 982/1990] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:29 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:29.380574 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a46224-cb5a-4f6f-89ec-2a2011bbeef6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:29.399640 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a46224-cb5a-4f6f-89ec-2a2011bbeef6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:29.406033 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdee390-7c32-41c6-b6ee-ae559782c83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:29.407253 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdee390-7c32-41c6-b6ee-ae559782c83b admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:29.407686 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdee390-7c32-41c6-b6ee-ae559782c83b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:29.408109 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdee390-7c32-41c6-b6ee-ae559782c83b admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:29.409069 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdee390-7c32-41c6-b6ee-ae559782c83b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:29.410940 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdee390-7c32-41c6-b6ee-ae559782c83b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:29.430180 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1009/1991] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:29 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:29.434589 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ea21d-0bdf-4655-b158-72454a3ee69f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:29.447448 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5ea21d-0bdf-4655-b158-72454a3ee69f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:29.458246 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c5a2b-156c-4839-ad94-8c488e400605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:29.459174 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c5a2b-156c-4839-ad94-8c488e400605 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:29.459454 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c5a2b-156c-4839-ad94-8c488e400605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:29.459726 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0c5a2b-156c-4839-ad94-8c488e400605 admin admin] PATH_INFO: `/v3/projects/95c081731b4c4b27b47bf098c89e5e2b/users/c5a4d93f070e4653a69d102b0a679dad/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:29.511415 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0c5a2b-156c-4839-ad94-8c488e400605 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95c081731b4c4b27b47bf098c89e5e2b, user_id=c5a4d93f070e4653a69d102b0a679dad, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:29.516020 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0c5a2b-156c-4839-ad94-8c488e400605 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:29.536205 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 983/1992] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:29 2022] PUT /identity/v3/projects/95c081731b4c4b27b47bf098c89e5e2b/users/c5a4d93f070e4653a69d102b0a679dad/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:29.548537 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e6d144-4474-41d1-8290-1dc1c24f5ba2 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:29.548833 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e6d144-4474-41d1-8290-1dc1c24f5ba2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:29.549113 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e6d144-4474-41d1-8290-1dc1c24f5ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:29.587312 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b3e6d144-4474-41d1-8290-1dc1c24f5ba2 None None] MFA Rules not processed for user `c5a4d93f070e4653a69d102b0a679dad`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:29.589568 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b3e6d144-4474-41d1-8290-1dc1c24f5ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:29.682011 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1010/1993] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:29 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:29.800858 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c5a2b-156c-4839-ad94-8c488e400605 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:29.802613 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cd0c5a2b-156c-4839-ad94-8c488e400605 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:29.865761 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0c5a2b-156c-4839-ad94-8c488e400605 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:29.871492 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c509cf6b-4fd2-48b6-b7cc-20f202b72436 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:29.872242 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c509cf6b-4fd2-48b6-b7cc-20f202b72436 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:29.872602 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c509cf6b-4fd2-48b6-b7cc-20f202b72436 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:29.872933 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c509cf6b-4fd2-48b6-b7cc-20f202b72436 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:29.883076 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c509cf6b-4fd2-48b6-b7cc-20f202b72436 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:29.885318 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c509cf6b-4fd2-48b6-b7cc-20f202b72436 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:29.897091 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 984/1994] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:29 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:29.929844 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e6d144-4474-41d1-8290-1dc1c24f5ba2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:30.052815 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e6d144-4474-41d1-8290-1dc1c24f5ba2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:30.059292 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9689fac8-b96c-4151-ae11-12518051d381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:30.060184 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689fac8-b96c-4151-ae11-12518051d381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:30.060463 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689fac8-b96c-4151-ae11-12518051d381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:30.060735 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9689fac8-b96c-4151-ae11-12518051d381 admin admin] PATH_INFO: `/v3/users/c5a4d93f070e4653a69d102b0a679dad` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:30.061942 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9689fac8-b96c-4151-ae11-12518051d381 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5a4d93f070e4653a69d102b0a679dad)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:30.063961 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9689fac8-b96c-4151-ae11-12518051d381 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:30.121957 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9689fac8-b96c-4151-ae11-12518051d381 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5a4d93f070e4653a69d102b0a679dad'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:30.123387 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:30.131713 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9689fac8-b96c-4151-ae11-12518051d381 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5a4d93f070e4653a69d102b0a679dad'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:30.133983 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9689fac8-b96c-4151-ae11-12518051d381 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5a4d93f070e4653a69d102b0a679dad'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:30.158067 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9689fac8-b96c-4151-ae11-12518051d381 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5a4d93f070e4653a69d102b0a679dad'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:30.159435 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9689fac8-b96c-4151-ae11-12518051d381 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5a4d93f070e4653a69d102b0a679dad'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:30.186339 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1011/1995] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:15:29 2022] DELETE /identity/v3/users/c5a4d93f070e4653a69d102b0a679dad => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:30.190971 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b4eaa-a9f7-411f-9f62-48456030f7a0 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:30.191125 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b4eaa-a9f7-411f-9f62-48456030f7a0 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:30.191776 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b4eaa-a9f7-411f-9f62-48456030f7a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:30.218096 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f27b4eaa-a9f7-411f-9f62-48456030f7a0 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:30.220317 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f27b4eaa-a9f7-411f-9f62-48456030f7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:30.279918 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:30.291375 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 985/1996] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:30.300819 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9689fac8-b96c-4151-ae11-12518051d381 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:30.302425 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9689fac8-b96c-4151-ae11-12518051d381 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:30.374409 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9689fac8-b96c-4151-ae11-12518051d381 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:30.378456 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eabbc85-3461-4e3d-b432-81dc4fcd8aa6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:30.379228 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eabbc85-3461-4e3d-b432-81dc4fcd8aa6 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:30.379589 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eabbc85-3461-4e3d-b432-81dc4fcd8aa6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:30.379942 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eabbc85-3461-4e3d-b432-81dc4fcd8aa6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:30.390692 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eabbc85-3461-4e3d-b432-81dc4fcd8aa6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:30.392446 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eabbc85-3461-4e3d-b432-81dc4fcd8aa6 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:30.403826 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1012/1997] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:30.754079 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b4eaa-a9f7-411f-9f62-48456030f7a0 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:30.755808 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f27b4eaa-a9f7-411f-9f62-48456030f7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:30.769273 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b4eaa-a9f7-411f-9f62-48456030f7a0 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:30.775938 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6170147d-fb65-4822-94bb-ed16a9e65016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:30.777394 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170147d-fb65-4822-94bb-ed16a9e65016 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:30.777680 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170147d-fb65-4822-94bb-ed16a9e65016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:30.778008 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6170147d-fb65-4822-94bb-ed16a9e65016 admin admin] PATH_INFO: `/v3/projects/95c081731b4c4b27b47bf098c89e5e2b` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:30.779387 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6170147d-fb65-4822-94bb-ed16a9e65016 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95c081731b4c4b27b47bf098c89e5e2b)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:30.781704 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6170147d-fb65-4822-94bb-ed16a9e65016 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:30.851149 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6170147d-fb65-4822-94bb-ed16a9e65016 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95c081731b4c4b27b47bf098c89e5e2b', 'request_id': 'req-6170147d-fb65-4822-94bb-ed16a9e65016'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:30.861885 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6170147d-fb65-4822-94bb-ed16a9e65016 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95c081731b4c4b27b47bf098c89e5e2b', 'request_id': 'req-6170147d-fb65-4822-94bb-ed16a9e65016'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:30.865133 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6170147d-fb65-4822-94bb-ed16a9e65016 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95c081731b4c4b27b47bf098c89e5e2b', 'request_id': 'req-6170147d-fb65-4822-94bb-ed16a9e65016'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:30.880058 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6170147d-fb65-4822-94bb-ed16a9e65016 admin admin] The token cache is being invalidate because project 95c081731b4c4b27b47bf098c89e5e2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:15:30.880703 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6170147d-fb65-4822-94bb-ed16a9e65016 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:30.881698 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-6170147d-fb65-4822-94bb-ed16a9e65016 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:30.883753 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 986/1998] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:15:30 2022] DELETE /identity/v3/projects/95c081731b4c4b27b47bf098c89e5e2b => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:30.888554 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a53855-d315-42ce-99ea-660865843389 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:30.888846 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a53855-d315-42ce-99ea-660865843389 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:30.889129 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a53855-d315-42ce-99ea-660865843389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:30.919826 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c9a53855-d315-42ce-99ea-660865843389 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:30.921235 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c9a53855-d315-42ce-99ea-660865843389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:30.994963 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1013/1999] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:31.002617 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52505e58-9e4e-48a8-9965-282e863dd009 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:31.003005 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52505e58-9e4e-48a8-9965-282e863dd009 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:31.003293 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52505e58-9e4e-48a8-9965-282e863dd009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:31.033065 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-52505e58-9e4e-48a8-9965-282e863dd009 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:31.034497 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-52505e58-9e4e-48a8-9965-282e863dd009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:31.044303 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 987/2000] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:31.048539 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a53855-d315-42ce-99ea-660865843389 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:31.061081 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a53855-d315-42ce-99ea-660865843389 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:31.064610 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf92fc00-7932-4de6-b7b0-9b708a3a8a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:31.065350 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92fc00-7932-4de6-b7b0-9b708a3a8a7a admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:31.065859 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92fc00-7932-4de6-b7b0-9b708a3a8a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:31.066202 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92fc00-7932-4de6-b7b0-9b708a3a8a7a admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:31.067125 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf92fc00-7932-4de6-b7b0-9b708a3a8a7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:15:31.067583 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf92fc00-7932-4de6-b7b0-9b708a3a8a7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:31.069274 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf92fc00-7932-4de6-b7b0-9b708a3a8a7a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:31.085624 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1014/2001] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:15:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:15:31.090707 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52505e58-9e4e-48a8-9965-282e863dd009 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:31.103253 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52505e58-9e4e-48a8-9965-282e863dd009 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:31.109175 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6d3eb4-8b78-4d88-90fc-abc215750694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:31.109978 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d3eb4-8b78-4d88-90fc-abc215750694 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:31.110415 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d3eb4-8b78-4d88-90fc-abc215750694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:31.110784 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d3eb4-8b78-4d88-90fc-abc215750694 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:31.111614 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6d3eb4-8b78-4d88-90fc-abc215750694 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:31.113965 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6d3eb4-8b78-4d88-90fc-abc215750694 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:31.146299 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 988/2002] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:15:31 2022] POST /identity/v3/projects => generated 397 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:31.150790 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf92fc00-7932-4de6-b7b0-9b708a3a8a7a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:31.167932 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf92fc00-7932-4de6-b7b0-9b708a3a8a7a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:31.243473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:31.258286 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf369761-df80-4a13-9442-f9db9eccbe09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:31.259767 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf369761-df80-4a13-9442-f9db9eccbe09 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:31.262825 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf369761-df80-4a13-9442-f9db9eccbe09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:31.263380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf369761-df80-4a13-9442-f9db9eccbe09 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:31.265012 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf369761-df80-4a13-9442-f9db9eccbe09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:31.266936 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf369761-df80-4a13-9442-f9db9eccbe09 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:31.296981 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1015/2003] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:15:31 2022] POST /identity/v3/users => generated 299 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:31.300901 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6d3eb4-8b78-4d88-90fc-abc215750694 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:31.319039 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6d3eb4-8b78-4d88-90fc-abc215750694 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:31.327156 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31046311-e1ac-483b-ae0a-19de4ffb665b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:31.327976 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31046311-e1ac-483b-ae0a-19de4ffb665b admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:31.328260 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31046311-e1ac-483b-ae0a-19de4ffb665b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:31.328540 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31046311-e1ac-483b-ae0a-19de4ffb665b admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:31.329736 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31046311-e1ac-483b-ae0a-19de4ffb665b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:31.331167 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31046311-e1ac-483b-ae0a-19de4ffb665b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:31.344481 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 989/2004] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:31.348736 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf369761-df80-4a13-9442-f9db9eccbe09 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:31.361094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf369761-df80-4a13-9442-f9db9eccbe09 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:31.369237 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899fdcfa-16ed-4021-af88-843ba30a8cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:31.369237 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899fdcfa-16ed-4021-af88-843ba30a8cf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:31.369237 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899fdcfa-16ed-4021-af88-843ba30a8cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:31.369237 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899fdcfa-16ed-4021-af88-843ba30a8cf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:31.370400 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899fdcfa-16ed-4021-af88-843ba30a8cf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:31.378454 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899fdcfa-16ed-4021-af88-843ba30a8cf1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:31.406331 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1016/2005] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:31.410475 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31046311-e1ac-483b-ae0a-19de4ffb665b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:31.422883 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31046311-e1ac-483b-ae0a-19de4ffb665b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:31.429843 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ab9eb-0508-4770-8abb-2573b008f1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:31.430796 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ab9eb-0508-4770-8abb-2573b008f1bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:31.431465 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ab9eb-0508-4770-8abb-2573b008f1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:31.434911 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0ab9eb-0508-4770-8abb-2573b008f1bf admin admin] PATH_INFO: `/v3/projects/1159939967494e57aabf510ed7c353ce/users/b72ccd82e12c477eb9c4ea151b32ba86/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:31.451891 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0ab9eb-0508-4770-8abb-2573b008f1bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=1159939967494e57aabf510ed7c353ce, user_id=b72ccd82e12c477eb9c4ea151b32ba86, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:31.455032 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0ab9eb-0508-4770-8abb-2573b008f1bf admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:31.475172 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 990/2006] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:31 2022] PUT /identity/v3/projects/1159939967494e57aabf510ed7c353ce/users/b72ccd82e12c477eb9c4ea151b32ba86/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:15:31.486231 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b4b61-1b13-434c-b513-f4854531878f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:31.486521 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b4b61-1b13-434c-b513-f4854531878f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:31.486799 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64b4b61-1b13-434c-b513-f4854531878f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:31.531284 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a64b4b61-1b13-434c-b513-f4854531878f None None] MFA Rules not processed for user `b72ccd82e12c477eb9c4ea151b32ba86`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:31.532925 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a64b4b61-1b13-434c-b513-f4854531878f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:31.615533 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1017/2007] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:31 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:31.627943 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ab9eb-0508-4770-8abb-2573b008f1bf admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:31.630130 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4c0ab9eb-0508-4770-8abb-2573b008f1bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:31.715943 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0ab9eb-0508-4770-8abb-2573b008f1bf admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:31.720060 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1249a59b-55b2-46ad-820b-81e8b86f2984 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:31.720869 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1249a59b-55b2-46ad-820b-81e8b86f2984 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:31.721230 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1249a59b-55b2-46ad-820b-81e8b86f2984 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:31.721579 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1249a59b-55b2-46ad-820b-81e8b86f2984 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:31.730422 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1249a59b-55b2-46ad-820b-81e8b86f2984 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:31.731979 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1249a59b-55b2-46ad-820b-81e8b86f2984 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:31.744250 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 991/2008] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:31 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:31.785562 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64b4b61-1b13-434c-b513-f4854531878f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:31.786990 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a64b4b61-1b13-434c-b513-f4854531878f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:31.872311 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64b4b61-1b13-434c-b513-f4854531878f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:31.888180 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af702b5e-079a-4b44-b7d1-890381b37273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:31.889239 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af702b5e-079a-4b44-b7d1-890381b37273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:31.889651 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af702b5e-079a-4b44-b7d1-890381b37273 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:31.890043 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af702b5e-079a-4b44-b7d1-890381b37273 admin admin] PATH_INFO: `/v3/users/bc060e442609458d8c27c404d598906a` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:31.892010 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af702b5e-079a-4b44-b7d1-890381b37273 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc060e442609458d8c27c404d598906a)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:31.894398 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af702b5e-079a-4b44-b7d1-890381b37273 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:31.994236 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-af702b5e-079a-4b44-b7d1-890381b37273 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc060e442609458d8c27c404d598906a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:32.004745 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-af702b5e-079a-4b44-b7d1-890381b37273 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc060e442609458d8c27c404d598906a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:32.006469 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-af702b5e-079a-4b44-b7d1-890381b37273 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc060e442609458d8c27c404d598906a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:32.016523 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-af702b5e-079a-4b44-b7d1-890381b37273 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc060e442609458d8c27c404d598906a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:32.017369 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-af702b5e-079a-4b44-b7d1-890381b37273 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc060e442609458d8c27c404d598906a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:32.020059 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:32.042405 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1018/2009] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:15:31 2022] DELETE /identity/v3/users/bc060e442609458d8c27c404d598906a => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:32.047190 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3cd063-e813-4086-90e2-3b29b02eba50 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:32.047682 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3cd063-e813-4086-90e2-3b29b02eba50 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:32.047772 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3cd063-e813-4086-90e2-3b29b02eba50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:32.068643 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-db3cd063-e813-4086-90e2-3b29b02eba50 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:32.069882 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-db3cd063-e813-4086-90e2-3b29b02eba50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:32.087173 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:32.177424 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 992/2010] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:32.185560 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af702b5e-079a-4b44-b7d1-890381b37273 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:32.187370 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-af702b5e-079a-4b44-b7d1-890381b37273 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:32.267825 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af702b5e-079a-4b44-b7d1-890381b37273 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:32.274319 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb41115-c751-497a-a685-a267167d35e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:32.275321 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb41115-c751-497a-a685-a267167d35e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:32.275709 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb41115-c751-497a-a685-a267167d35e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:32.276610 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb41115-c751-497a-a685-a267167d35e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:32.290775 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb41115-c751-497a-a685-a267167d35e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:32.294455 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb41115-c751-497a-a685-a267167d35e6 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:32.309118 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1019/2011] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:15:32.612991 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3cd063-e813-4086-90e2-3b29b02eba50 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:32.614584 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb41115-c751-497a-a685-a267167d35e6 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:32.614707 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-db3cd063-e813-4086-90e2-3b29b02eba50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:32.616198 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8bb41115-c751-497a-a685-a267167d35e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:32.626999 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3cd063-e813-4086-90e2-3b29b02eba50 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:32.630870 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb41115-c751-497a-a685-a267167d35e6 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:32.632850 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b020a-f32d-4899-aa98-f862f89eb439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:32.633931 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b020a-f32d-4899-aa98-f862f89eb439 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:32.634372 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b020a-f32d-4899-aa98-f862f89eb439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:32.634814 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b020a-f32d-4899-aa98-f862f89eb439 admin admin] PATH_INFO: `/v3/projects/30a492c71c0c4afd927bc0ad8c4c46fc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:32.636205 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c967264-1196-4fb8-b4ee-d819eb48c044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:32.636850 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b020a-f32d-4899-aa98-f862f89eb439 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30a492c71c0c4afd927bc0ad8c4c46fc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:32.637082 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c967264-1196-4fb8-b4ee-d819eb48c044 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:32.637370 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c967264-1196-4fb8-b4ee-d819eb48c044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:32.637805 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c967264-1196-4fb8-b4ee-d819eb48c044 admin admin] PATH_INFO: `/v3/users/f00d5ae33cb642979533f6cd5c27a02d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:32.639377 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b020a-f32d-4899-aa98-f862f89eb439 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:32.639490 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c967264-1196-4fb8-b4ee-d819eb48c044 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f00d5ae33cb642979533f6cd5c27a02d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:32.641345 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c967264-1196-4fb8-b4ee-d819eb48c044 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:32.693270 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-dc2b020a-f32d-4899-aa98-f862f89eb439 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30a492c71c0c4afd927bc0ad8c4c46fc', 'request_id': 'req-dc2b020a-f32d-4899-aa98-f862f89eb439'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:32.700458 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-dc2b020a-f32d-4899-aa98-f862f89eb439 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30a492c71c0c4afd927bc0ad8c4c46fc', 'request_id': 'req-dc2b020a-f32d-4899-aa98-f862f89eb439'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:32.702691 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5c967264-1196-4fb8-b4ee-d819eb48c044 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f00d5ae33cb642979533f6cd5c27a02d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:32.705801 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-dc2b020a-f32d-4899-aa98-f862f89eb439 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30a492c71c0c4afd927bc0ad8c4c46fc', 'request_id': 'req-dc2b020a-f32d-4899-aa98-f862f89eb439'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:32.711856 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5c967264-1196-4fb8-b4ee-d819eb48c044 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f00d5ae33cb642979533f6cd5c27a02d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:32.712871 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5c967264-1196-4fb8-b4ee-d819eb48c044 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f00d5ae33cb642979533f6cd5c27a02d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:32.723685 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-dc2b020a-f32d-4899-aa98-f862f89eb439 admin admin] The token cache is being invalidate because project 30a492c71c0c4afd927bc0ad8c4c46fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:15:32.723851 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5c967264-1196-4fb8-b4ee-d819eb48c044 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f00d5ae33cb642979533f6cd5c27a02d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:32.723963 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-dc2b020a-f32d-4899-aa98-f862f89eb439 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:32.724496 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5c967264-1196-4fb8-b4ee-d819eb48c044 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f00d5ae33cb642979533f6cd5c27a02d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:32.724651 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-dc2b020a-f32d-4899-aa98-f862f89eb439 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:32.729824 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 993/2012] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:15:32 2022] DELETE /identity/v3/projects/30a492c71c0c4afd927bc0ad8c4c46fc => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:32.731607 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b020a-f32d-4899-aa98-f862f89eb439 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:32.733328 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-dc2b020a-f32d-4899-aa98-f862f89eb439 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:32.749273 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1020/2013] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:15:32 2022] DELETE /identity/v3/users/f00d5ae33cb642979533f6cd5c27a02d => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:15:32.753649 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db4d7b-57f4-4d05-916a-e5e3817629f0 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:32.754045 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db4d7b-57f4-4d05-916a-e5e3817629f0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:32.754437 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db4d7b-57f4-4d05-916a-e5e3817629f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:32.774518 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-64db4d7b-57f4-4d05-916a-e5e3817629f0 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:32.775774 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-64db4d7b-57f4-4d05-916a-e5e3817629f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:32.803370 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b020a-f32d-4899-aa98-f862f89eb439 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:32.828662 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:32.843389 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1021/2014] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:15:32.850339 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64db4d7b-57f4-4d05-916a-e5e3817629f0 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:32.851687 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-64db4d7b-57f4-4d05-916a-e5e3817629f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:32.852674 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:32.865450 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389cb590-b9bc-49ef-b8d9-ce320af2580f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:32.866324 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389cb590-b9bc-49ef-b8d9-ce320af2580f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:32.866604 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389cb590-b9bc-49ef-b8d9-ce320af2580f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:32.866903 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389cb590-b9bc-49ef-b8d9-ce320af2580f admin admin] PATH_INFO: `/v3/users/00b72cc486eb411193e3dfba88f192d2` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:32.868095 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389cb590-b9bc-49ef-b8d9-ce320af2580f admin admin] RBAC: Authorizing `identity:delete_user(user_id=00b72cc486eb411193e3dfba88f192d2)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:32.870136 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389cb590-b9bc-49ef-b8d9-ce320af2580f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:32.917534 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64db4d7b-57f4-4d05-916a-e5e3817629f0 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:32.921967 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a53bd1f-86ea-43c8-8f8f-60c4bec8e31a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:32.922516 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-389cb590-b9bc-49ef-b8d9-ce320af2580f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00b72cc486eb411193e3dfba88f192d2'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:32.922731 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53bd1f-86ea-43c8-8f8f-60c4bec8e31a service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:32.923021 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53bd1f-86ea-43c8-8f8f-60c4bec8e31a service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:32.923336 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53bd1f-86ea-43c8-8f8f-60c4bec8e31a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:32.931909 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-389cb590-b9bc-49ef-b8d9-ce320af2580f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00b72cc486eb411193e3dfba88f192d2'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:32.936140 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a53bd1f-86ea-43c8-8f8f-60c4bec8e31a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:32.937806 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a53bd1f-86ea-43c8-8f8f-60c4bec8e31a service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:32.940945 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-389cb590-b9bc-49ef-b8d9-ce320af2580f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00b72cc486eb411193e3dfba88f192d2'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:32.941845 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-389cb590-b9bc-49ef-b8d9-ce320af2580f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00b72cc486eb411193e3dfba88f192d2'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:32.942505 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-389cb590-b9bc-49ef-b8d9-ce320af2580f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00b72cc486eb411193e3dfba88f192d2'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:32.949298 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1022/2015] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:15:32.961505 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 994/2016] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:15:32 2022] DELETE /identity/v3/users/00b72cc486eb411193e3dfba88f192d2 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:33.245428 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a53bd1f-86ea-43c8-8f8f-60c4bec8e31a service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:33.247116 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2a53bd1f-86ea-43c8-8f8f-60c4bec8e31a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:33.266707 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a53bd1f-86ea-43c8-8f8f-60c4bec8e31a service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:33.280643 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389cb590-b9bc-49ef-b8d9-ce320af2580f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:33.283488 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-389cb590-b9bc-49ef-b8d9-ce320af2580f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:33.351484 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3967c258-38b3-4129-82d7-6c99ef85b101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:33.352423 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3967c258-38b3-4129-82d7-6c99ef85b101 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:33.352715 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3967c258-38b3-4129-82d7-6c99ef85b101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:33.353020 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3967c258-38b3-4129-82d7-6c99ef85b101 admin admin] PATH_INFO: `/v3/projects/c3c4757ffbbb4f5ba3cda979c84a20df` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:33.354341 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3967c258-38b3-4129-82d7-6c99ef85b101 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3c4757ffbbb4f5ba3cda979c84a20df)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:33.356437 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3967c258-38b3-4129-82d7-6c99ef85b101 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:33.368941 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389cb590-b9bc-49ef-b8d9-ce320af2580f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:33.377955 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eba6eb-661e-47eb-a33c-ee3a3d3885d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:33.379476 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eba6eb-661e-47eb-a33c-ee3a3d3885d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:33.379833 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eba6eb-661e-47eb-a33c-ee3a3d3885d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:33.380402 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eba6eb-661e-47eb-a33c-ee3a3d3885d1 admin admin] PATH_INFO: `/v3/projects/ec8c19b199db4ae3aeca0c2aeb030fd5` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:33.383061 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eba6eb-661e-47eb-a33c-ee3a3d3885d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec8c19b199db4ae3aeca0c2aeb030fd5)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:33.385064 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eba6eb-661e-47eb-a33c-ee3a3d3885d1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:33.414347 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3967c258-38b3-4129-82d7-6c99ef85b101 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3c4757ffbbb4f5ba3cda979c84a20df', 'request_id': 'req-3967c258-38b3-4129-82d7-6c99ef85b101'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:33.422894 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3967c258-38b3-4129-82d7-6c99ef85b101 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3c4757ffbbb4f5ba3cda979c84a20df', 'request_id': 'req-3967c258-38b3-4129-82d7-6c99ef85b101'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:33.429042 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3967c258-38b3-4129-82d7-6c99ef85b101 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3c4757ffbbb4f5ba3cda979c84a20df', 'request_id': 'req-3967c258-38b3-4129-82d7-6c99ef85b101'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:33.440778 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3967c258-38b3-4129-82d7-6c99ef85b101 admin admin] The token cache is being invalidate because project c3c4757ffbbb4f5ba3cda979c84a20df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:15:33.441069 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3967c258-38b3-4129-82d7-6c99ef85b101 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:33.441720 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3967c258-38b3-4129-82d7-6c99ef85b101 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:33.443319 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1023/2017] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:15:33 2022] DELETE /identity/v3/projects/c3c4757ffbbb4f5ba3cda979c84a20df => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:33.447207 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3967c258-38b3-4129-82d7-6c99ef85b101 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:33.449539 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3967c258-38b3-4129-82d7-6c99ef85b101 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:33.449688 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e5eba6eb-661e-47eb-a33c-ee3a3d3885d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec8c19b199db4ae3aeca0c2aeb030fd5', 'request_id': 'req-e5eba6eb-661e-47eb-a33c-ee3a3d3885d1'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:33.455329 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e5eba6eb-661e-47eb-a33c-ee3a3d3885d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec8c19b199db4ae3aeca0c2aeb030fd5', 'request_id': 'req-e5eba6eb-661e-47eb-a33c-ee3a3d3885d1'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:33.461695 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e5eba6eb-661e-47eb-a33c-ee3a3d3885d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec8c19b199db4ae3aeca0c2aeb030fd5', 'request_id': 'req-e5eba6eb-661e-47eb-a33c-ee3a3d3885d1'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:33.472869 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:33.477472 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e5eba6eb-661e-47eb-a33c-ee3a3d3885d1 admin admin] The token cache is being invalidate because project ec8c19b199db4ae3aeca0c2aeb030fd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:15:33.477831 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e5eba6eb-661e-47eb-a33c-ee3a3d3885d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:33.478843 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e5eba6eb-661e-47eb-a33c-ee3a3d3885d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:33.481818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 995/2018] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:15:33 2022] DELETE /identity/v3/projects/ec8c19b199db4ae3aeca0c2aeb030fd5 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:33.486889 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda349c9-5862-4396-9189-597e0de913bd None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:33.487194 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda349c9-5862-4396-9189-597e0de913bd None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:33.487346 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:33.487501 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda349c9-5862-4396-9189-597e0de913bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:33.507830 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-bda349c9-5862-4396-9189-597e0de913bd None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:33.509115 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bda349c9-5862-4396-9189-597e0de913bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:33.519918 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3967c258-38b3-4129-82d7-6c99ef85b101 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:33.570324 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 996/2019] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:15:33.577794 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dfb5c3-fdd5-4b96-b379-6953f30c3a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:33.578734 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dfb5c3-fdd5-4b96-b379-6953f30c3a41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:33.578889 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d963e-e95c-4a42-82d8-4161b5b5c010 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:33.579098 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d963e-e95c-4a42-82d8-4161b5b5c010 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:33.579199 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dfb5c3-fdd5-4b96-b379-6953f30c3a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:33.579351 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881d963e-e95c-4a42-82d8-4161b5b5c010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:33.579704 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dfb5c3-fdd5-4b96-b379-6953f30c3a41 admin admin] PATH_INFO: `/v3/users/a2a6874387fd460984cb8e84856f511f` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:33.580991 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dfb5c3-fdd5-4b96-b379-6953f30c3a41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2a6874387fd460984cb8e84856f511f)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:33.583072 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dfb5c3-fdd5-4b96-b379-6953f30c3a41 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:33.597753 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-881d963e-e95c-4a42-82d8-4161b5b5c010 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:33.598970 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-881d963e-e95c-4a42-82d8-4161b5b5c010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:33.607773 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 997/2020] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:33.612000 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881d963e-e95c-4a42-82d8-4161b5b5c010 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:33.625628 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881d963e-e95c-4a42-82d8-4161b5b5c010 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:33.631399 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebddb1f7-084f-4d5b-97be-4cb012e9a4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:33.632184 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebddb1f7-084f-4d5b-97be-4cb012e9a4bf admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:33.632610 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebddb1f7-084f-4d5b-97be-4cb012e9a4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:33.633041 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebddb1f7-084f-4d5b-97be-4cb012e9a4bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:33.633981 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebddb1f7-084f-4d5b-97be-4cb012e9a4bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:15:33.634434 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebddb1f7-084f-4d5b-97be-4cb012e9a4bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:33.636468 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebddb1f7-084f-4d5b-97be-4cb012e9a4bf admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:33.639149 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-35dfb5c3-fdd5-4b96-b379-6953f30c3a41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2a6874387fd460984cb8e84856f511f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:33.647638 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-35dfb5c3-fdd5-4b96-b379-6953f30c3a41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2a6874387fd460984cb8e84856f511f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:33.648853 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-35dfb5c3-fdd5-4b96-b379-6953f30c3a41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2a6874387fd460984cb8e84856f511f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:33.649977 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 998/2021] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:15:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:15:33.654919 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebddb1f7-084f-4d5b-97be-4cb012e9a4bf admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:33.658105 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-35dfb5c3-fdd5-4b96-b379-6953f30c3a41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2a6874387fd460984cb8e84856f511f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:33.658890 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-35dfb5c3-fdd5-4b96-b379-6953f30c3a41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2a6874387fd460984cb8e84856f511f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:33.666307 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebddb1f7-084f-4d5b-97be-4cb012e9a4bf admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:33.671845 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011498b3-a5d4-4084-bcf1-dd201f7659af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:33.672491 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011498b3-a5d4-4084-bcf1-dd201f7659af admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:33.672771 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011498b3-a5d4-4084-bcf1-dd201f7659af admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:33.673248 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011498b3-a5d4-4084-bcf1-dd201f7659af admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:33.673724 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011498b3-a5d4-4084-bcf1-dd201f7659af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:33.675662 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011498b3-a5d4-4084-bcf1-dd201f7659af admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:33.677139 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1024/2022] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:15:33 2022] DELETE /identity/v3/users/a2a6874387fd460984cb8e84856f511f => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:15:33.708520 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 999/2023] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:15:33 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:33.713026 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dfb5c3-fdd5-4b96-b379-6953f30c3a41 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:33.714451 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-35dfb5c3-fdd5-4b96-b379-6953f30c3a41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:33.787387 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:33.793472 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dfb5c3-fdd5-4b96-b379-6953f30c3a41 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:33.866588 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a5535-2db8-4a1f-86e0-cc7b70f3c9b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:33.867317 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a5535-2db8-4a1f-86e0-cc7b70f3c9b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:33.867633 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a5535-2db8-4a1f-86e0-cc7b70f3c9b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:33.867934 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a5535-2db8-4a1f-86e0-cc7b70f3c9b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:33.868692 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422a5535-2db8-4a1f-86e0-cc7b70f3c9b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:33.871384 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422a5535-2db8-4a1f-86e0-cc7b70f3c9b0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:33.895208 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1025/2024] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:15:33 2022] POST /identity/v3/users => generated 287 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:15:33.899420 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011498b3-a5d4-4084-bcf1-dd201f7659af admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:33.910176 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011498b3-a5d4-4084-bcf1-dd201f7659af admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:33.914892 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a961f50d-c836-4436-a3e5-fbace8389cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:33.915547 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a961f50d-c836-4436-a3e5-fbace8389cf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:33.915836 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a961f50d-c836-4436-a3e5-fbace8389cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:33.916121 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a961f50d-c836-4436-a3e5-fbace8389cf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:33.917679 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a961f50d-c836-4436-a3e5-fbace8389cf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:33.919333 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a961f50d-c836-4436-a3e5-fbace8389cf9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:33.930298 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1000/2025] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:33.935138 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a5535-2db8-4a1f-86e0-cc7b70f3c9b0 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:33.945413 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a5535-2db8-4a1f-86e0-cc7b70f3c9b0 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:33.950221 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf457350-d5a1-43d6-ad7e-d025103cad68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:33.950933 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf457350-d5a1-43d6-ad7e-d025103cad68 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:33.951225 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf457350-d5a1-43d6-ad7e-d025103cad68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:33.951503 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf457350-d5a1-43d6-ad7e-d025103cad68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:33.952257 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf457350-d5a1-43d6-ad7e-d025103cad68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:33.954214 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf457350-d5a1-43d6-ad7e-d025103cad68 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:33.964964 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1026/2026] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:33.970029 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a961f50d-c836-4436-a3e5-fbace8389cf9 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:33.970633 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf457350-d5a1-43d6-ad7e-d025103cad68 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:33.972433 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bf457350-d5a1-43d6-ad7e-d025103cad68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:33.983630 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a961f50d-c836-4436-a3e5-fbace8389cf9 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:33.989760 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b26716c-1e64-46ea-a8c0-07e72421c233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:33.990785 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26716c-1e64-46ea-a8c0-07e72421c233 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:33.991692 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26716c-1e64-46ea-a8c0-07e72421c233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:33.991985 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b26716c-1e64-46ea-a8c0-07e72421c233 admin admin] PATH_INFO: `/v3/projects/88db59a73fc14dbbb1d52e252bf1899f/users/6456820135f2455b99106cf9df601324/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:33.992159 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf457350-d5a1-43d6-ad7e-d025103cad68 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:33.998783 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1714fbe0-292d-4778-8547-06796c43b61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:33.999621 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1714fbe0-292d-4778-8547-06796c43b61e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:33.999904 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1714fbe0-292d-4778-8547-06796c43b61e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:34.000244 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1714fbe0-292d-4778-8547-06796c43b61e admin admin] PATH_INFO: `/v3/projects/78c337fa58c04e3db006478434dbc624` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:34.001516 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1714fbe0-292d-4778-8547-06796c43b61e admin admin] RBAC: Authorizing `identity:delete_project(project_id=78c337fa58c04e3db006478434dbc624)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:34.004391 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1714fbe0-292d-4778-8547-06796c43b61e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:34.010242 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b26716c-1e64-46ea-a8c0-07e72421c233 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88db59a73fc14dbbb1d52e252bf1899f, user_id=6456820135f2455b99106cf9df601324, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:34.012883 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b26716c-1e64-46ea-a8c0-07e72421c233 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:34.027970 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1001/2027] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:33 2022] PUT /identity/v3/projects/88db59a73fc14dbbb1d52e252bf1899f/users/6456820135f2455b99106cf9df601324/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:34.044600 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd7ca30-d0d5-4443-b4b7-0e6dfb774a4a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:34.044926 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd7ca30-d0d5-4443-b4b7-0e6dfb774a4a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:34.045241 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd7ca30-d0d5-4443-b4b7-0e6dfb774a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:34.061949 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1714fbe0-292d-4778-8547-06796c43b61e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78c337fa58c04e3db006478434dbc624', 'request_id': 'req-1714fbe0-292d-4778-8547-06796c43b61e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:34.070546 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1714fbe0-292d-4778-8547-06796c43b61e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78c337fa58c04e3db006478434dbc624', 'request_id': 'req-1714fbe0-292d-4778-8547-06796c43b61e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:34.075754 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-acd7ca30-d0d5-4443-b4b7-0e6dfb774a4a None None] MFA Rules not processed for user `6456820135f2455b99106cf9df601324`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:34.077057 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-acd7ca30-d0d5-4443-b4b7-0e6dfb774a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:34.077802 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1714fbe0-292d-4778-8547-06796c43b61e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78c337fa58c04e3db006478434dbc624', 'request_id': 'req-1714fbe0-292d-4778-8547-06796c43b61e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:34.092653 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1714fbe0-292d-4778-8547-06796c43b61e admin admin] The token cache is being invalidate because project 78c337fa58c04e3db006478434dbc624 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:15:34.093201 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1714fbe0-292d-4778-8547-06796c43b61e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:34.093927 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1714fbe0-292d-4778-8547-06796c43b61e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:34.095519 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1027/2028] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:15:33 2022] DELETE /identity/v3/projects/78c337fa58c04e3db006478434dbc624 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:15:34.100370 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af820ae-0581-4157-8ca1-41be3736aacf None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:34.100754 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af820ae-0581-4157-8ca1-41be3736aacf None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:34.101031 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af820ae-0581-4157-8ca1-41be3736aacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:34.121353 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3af820ae-0581-4157-8ca1-41be3736aacf None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:34.122556 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3af820ae-0581-4157-8ca1-41be3736aacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:34.151574 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1002/2029] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:34.178349 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd7ca30-d0d5-4443-b4b7-0e6dfb774a4a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:34.180248 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-acd7ca30-d0d5-4443-b4b7-0e6dfb774a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:34.189080 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1028/2030] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:15:34.196351 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:34.199363 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355b37b-4eb8-41d2-b820-e708ebd4f198 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:34.199776 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355b37b-4eb8-41d2-b820-e708ebd4f198 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:34.200288 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355b37b-4eb8-41d2-b820-e708ebd4f198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:34.216569 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:34.223523 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a355b37b-4eb8-41d2-b820-e708ebd4f198 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:34.225661 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a355b37b-4eb8-41d2-b820-e708ebd4f198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:34.235636 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1029/2031] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:34.239818 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355b37b-4eb8-41d2-b820-e708ebd4f198 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:34.244030 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd7ca30-d0d5-4443-b4b7-0e6dfb774a4a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:34.248518 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e04096-c223-45ce-948c-0519534f9092 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:34.249085 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355b37b-4eb8-41d2-b820-e708ebd4f198 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:34.249593 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e04096-c223-45ce-948c-0519534f9092 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:34.249593 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e04096-c223-45ce-948c-0519534f9092 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:34.249777 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e04096-c223-45ce-948c-0519534f9092 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:34.253619 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7961c34a-c709-40af-9668-dd0a707a9741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:34.254263 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961c34a-c709-40af-9668-dd0a707a9741 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:34.254548 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961c34a-c709-40af-9668-dd0a707a9741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:34.254819 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961c34a-c709-40af-9668-dd0a707a9741 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:34.255596 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7961c34a-c709-40af-9668-dd0a707a9741 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:15:34.255880 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7961c34a-c709-40af-9668-dd0a707a9741 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:34.257462 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7961c34a-c709-40af-9668-dd0a707a9741 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:34.258584 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e04096-c223-45ce-948c-0519534f9092 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:34.260036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e04096-c223-45ce-948c-0519534f9092 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:34.270976 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1003/2032] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:34 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:34.272957 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1030/2033] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:15:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:15:34.279318 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e04096-c223-45ce-948c-0519534f9092 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:34.288612 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7961c34a-c709-40af-9668-dd0a707a9741 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:34.290279 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7961c34a-c709-40af-9668-dd0a707a9741 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:34.291363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e04096-c223-45ce-948c-0519534f9092 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:34.296594 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4221c9-4a6c-42c7-88c5-7cd944ba7f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:34.297273 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4221c9-4a6c-42c7-88c5-7cd944ba7f0a admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:34.297694 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4221c9-4a6c-42c7-88c5-7cd944ba7f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:34.297985 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4221c9-4a6c-42c7-88c5-7cd944ba7f0a admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:34.298686 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4221c9-4a6c-42c7-88c5-7cd944ba7f0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:34.300629 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4221c9-4a6c-42c7-88c5-7cd944ba7f0a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:34.302836 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7961c34a-c709-40af-9668-dd0a707a9741 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:34.307550 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f04643e-9b86-4014-a6c0-1ac661a9be40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:34.308459 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04643e-9b86-4014-a6c0-1ac661a9be40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:34.308737 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04643e-9b86-4014-a6c0-1ac661a9be40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:34.309019 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f04643e-9b86-4014-a6c0-1ac661a9be40 admin admin] PATH_INFO: `/v3/users/6456820135f2455b99106cf9df601324` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:34.310180 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f04643e-9b86-4014-a6c0-1ac661a9be40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6456820135f2455b99106cf9df601324)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:34.312136 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f04643e-9b86-4014-a6c0-1ac661a9be40 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:34.331030 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1004/2034] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:15:34 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:34.331984 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4221c9-4a6c-42c7-88c5-7cd944ba7f0a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:34.363152 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2f04643e-9b86-4014-a6c0-1ac661a9be40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6456820135f2455b99106cf9df601324'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:34.372551 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2f04643e-9b86-4014-a6c0-1ac661a9be40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6456820135f2455b99106cf9df601324'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:34.373255 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2f04643e-9b86-4014-a6c0-1ac661a9be40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6456820135f2455b99106cf9df601324'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:34.380477 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2f04643e-9b86-4014-a6c0-1ac661a9be40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6456820135f2455b99106cf9df601324'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:34.381304 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2f04643e-9b86-4014-a6c0-1ac661a9be40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6456820135f2455b99106cf9df601324'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:34.400760 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4221c9-4a6c-42c7-88c5-7cd944ba7f0a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:34.403194 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1031/2035] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:15:34 2022] DELETE /identity/v3/users/6456820135f2455b99106cf9df601324 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:15:34.407476 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08826cc-94e6-4c47-8502-71a444edae16 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:34.407765 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08826cc-94e6-4c47-8502-71a444edae16 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:34.408129 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08826cc-94e6-4c47-8502-71a444edae16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:34.428377 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b08826cc-94e6-4c47-8502-71a444edae16 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:34.429702 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b08826cc-94e6-4c47-8502-71a444edae16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:34.466192 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb689ffa-6d0d-4b76-b4b6-bd49569fa745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:34.467461 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb689ffa-6d0d-4b76-b4b6-bd49569fa745 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:34.467821 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb689ffa-6d0d-4b76-b4b6-bd49569fa745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:34.468422 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb689ffa-6d0d-4b76-b4b6-bd49569fa745 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:34.469290 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb689ffa-6d0d-4b76-b4b6-bd49569fa745 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:34.471328 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb689ffa-6d0d-4b76-b4b6-bd49569fa745 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:34.496908 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1005/2036] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:15:34 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:15:34.501234 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb689ffa-6d0d-4b76-b4b6-bd49569fa745 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:34.504205 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cb689ffa-6d0d-4b76-b4b6-bd49569fa745 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:34.512740 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1032/2037] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:15:34.517789 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb689ffa-6d0d-4b76-b4b6-bd49569fa745 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:34.522424 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08826cc-94e6-4c47-8502-71a444edae16 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:34.522682 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85302670-e56b-4b9d-b4ac-a4dfb4ec6227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:34.523431 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85302670-e56b-4b9d-b4ac-a4dfb4ec6227 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:34.523718 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85302670-e56b-4b9d-b4ac-a4dfb4ec6227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:34.523812 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b08826cc-94e6-4c47-8502-71a444edae16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:34.523992 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85302670-e56b-4b9d-b4ac-a4dfb4ec6227 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:34.524715 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85302670-e56b-4b9d-b4ac-a4dfb4ec6227 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:34.526911 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85302670-e56b-4b9d-b4ac-a4dfb4ec6227 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:34.537102 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1006/2038] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:34.540911 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85302670-e56b-4b9d-b4ac-a4dfb4ec6227 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:34.551015 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85302670-e56b-4b9d-b4ac-a4dfb4ec6227 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:34.555602 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d66706-2507-4b15-9e1b-ce4f7a9bba4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:34.556232 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d66706-2507-4b15-9e1b-ce4f7a9bba4f admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:34.556515 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d66706-2507-4b15-9e1b-ce4f7a9bba4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:34.556787 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d66706-2507-4b15-9e1b-ce4f7a9bba4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:34.558295 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d66706-2507-4b15-9e1b-ce4f7a9bba4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:34.559895 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d66706-2507-4b15-9e1b-ce4f7a9bba4f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:34.569814 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1007/2039] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:15:34.573748 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d66706-2507-4b15-9e1b-ce4f7a9bba4f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:34.582443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08826cc-94e6-4c47-8502-71a444edae16 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:34.583087 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d66706-2507-4b15-9e1b-ce4f7a9bba4f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:34.586203 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720e3715-610f-4b8b-b54a-ad2838c126d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:34.586743 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e3715-610f-4b8b-b54a-ad2838c126d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:34.587058 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e3715-610f-4b8b-b54a-ad2838c126d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:34.587341 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720e3715-610f-4b8b-b54a-ad2838c126d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:34.587590 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020c5b8b-d8dc-4f9f-9d7d-51ec82322b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:34.588622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c5b8b-d8dc-4f9f-9d7d-51ec82322b2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:34.588737 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c5b8b-d8dc-4f9f-9d7d-51ec82322b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:34.588996 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020c5b8b-d8dc-4f9f-9d7d-51ec82322b2d admin admin] PATH_INFO: `/v3/projects/c2edef81c7ce4f0ba0a52d30a926883c/users/4e9116d5ca37444983ed48243f4828ad/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:34.591966 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:34.597695 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720e3715-610f-4b8b-b54a-ad2838c126d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:34.599444 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720e3715-610f-4b8b-b54a-ad2838c126d7 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:34.606380 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020c5b8b-d8dc-4f9f-9d7d-51ec82322b2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2edef81c7ce4f0ba0a52d30a926883c, user_id=4e9116d5ca37444983ed48243f4828ad, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:34.609214 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020c5b8b-d8dc-4f9f-9d7d-51ec82322b2d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:34.610043 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1033/2040] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:15:34.622355 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1008/2041] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:34 2022] PUT /identity/v3/projects/c2edef81c7ce4f0ba0a52d30a926883c/users/4e9116d5ca37444983ed48243f4828ad/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:34.632399 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a37895-135e-4369-bd2e-fc210322dd10 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:34.632744 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a37895-135e-4369-bd2e-fc210322dd10 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:34.633032 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a37895-135e-4369-bd2e-fc210322dd10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:34.639560 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:34.663494 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-65a37895-135e-4369-bd2e-fc210322dd10 None None] MFA Rules not processed for user `4e9116d5ca37444983ed48243f4828ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:34.664705 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-65a37895-135e-4369-bd2e-fc210322dd10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:34.741694 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1034/2042] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:34 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:34.746642 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020c5b8b-d8dc-4f9f-9d7d-51ec82322b2d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:34.819901 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020c5b8b-d8dc-4f9f-9d7d-51ec82322b2d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:34.825100 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf482a7f-bace-49ce-a126-2fd9517f657d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:34.825757 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf482a7f-bace-49ce-a126-2fd9517f657d admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:34.826042 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf482a7f-bace-49ce-a126-2fd9517f657d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:34.826316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf482a7f-bace-49ce-a126-2fd9517f657d admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:34.827079 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf482a7f-bace-49ce-a126-2fd9517f657d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:34.828987 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf482a7f-bace-49ce-a126-2fd9517f657d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:34.859079 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1009/2043] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:15:34 2022] POST /identity/v3/projects => generated 377 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:34.862825 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a37895-135e-4369-bd2e-fc210322dd10 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:34.874007 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a37895-135e-4369-bd2e-fc210322dd10 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:34.911409 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf482a7f-bace-49ce-a126-2fd9517f657d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:34.914074 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bf482a7f-bace-49ce-a126-2fd9517f657d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:34.955511 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd07d39-40c0-46a2-8d70-7cb64d7bd5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:34.956181 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd07d39-40c0-46a2-8d70-7cb64d7bd5e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:34.956462 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd07d39-40c0-46a2-8d70-7cb64d7bd5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:34.956738 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd07d39-40c0-46a2-8d70-7cb64d7bd5e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:34.957436 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd07d39-40c0-46a2-8d70-7cb64d7bd5e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:34.959380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd07d39-40c0-46a2-8d70-7cb64d7bd5e0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:34.980500 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1035/2044] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:15:34 2022] POST /identity/v3/users => generated 289 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:15:34.984383 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd07d39-40c0-46a2-8d70-7cb64d7bd5e0 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:34.987606 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf482a7f-bace-49ce-a126-2fd9517f657d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:34.992550 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80286f9-b251-4cbe-b7d5-75c9db162eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:34.993530 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80286f9-b251-4cbe-b7d5-75c9db162eca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:34.993834 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80286f9-b251-4cbe-b7d5-75c9db162eca admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:34.994106 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80286f9-b251-4cbe-b7d5-75c9db162eca admin admin] PATH_INFO: `/v3/projects/88db59a73fc14dbbb1d52e252bf1899f` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:34.994446 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd07d39-40c0-46a2-8d70-7cb64d7bd5e0 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:34.995740 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80286f9-b251-4cbe-b7d5-75c9db162eca admin admin] RBAC: Authorizing `identity:delete_project(project_id=88db59a73fc14dbbb1d52e252bf1899f)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:34.997715 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80286f9-b251-4cbe-b7d5-75c9db162eca admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:35.001620 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e15ab5-0aae-46d2-a539-9e6d59600518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:35.002276 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e15ab5-0aae-46d2-a539-9e6d59600518 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:35.002555 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e15ab5-0aae-46d2-a539-9e6d59600518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:35.002828 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e15ab5-0aae-46d2-a539-9e6d59600518 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:35.003643 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e15ab5-0aae-46d2-a539-9e6d59600518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:35.005207 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e15ab5-0aae-46d2-a539-9e6d59600518 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:35.016282 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1036/2045] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:15:35.020457 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e15ab5-0aae-46d2-a539-9e6d59600518 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:35.031957 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e15ab5-0aae-46d2-a539-9e6d59600518 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:35.036682 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b697383-6040-453e-8313-8571f3f93a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:35.037333 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b697383-6040-453e-8313-8571f3f93a70 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:35.037620 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b697383-6040-453e-8313-8571f3f93a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:35.037900 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b697383-6040-453e-8313-8571f3f93a70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:35.038644 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b697383-6040-453e-8313-8571f3f93a70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:35.040261 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b697383-6040-453e-8313-8571f3f93a70 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:35.049266 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a80286f9-b251-4cbe-b7d5-75c9db162eca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88db59a73fc14dbbb1d52e252bf1899f', 'request_id': 'req-a80286f9-b251-4cbe-b7d5-75c9db162eca'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:35.057139 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1037/2046] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:15:35.057389 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a80286f9-b251-4cbe-b7d5-75c9db162eca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88db59a73fc14dbbb1d52e252bf1899f', 'request_id': 'req-a80286f9-b251-4cbe-b7d5-75c9db162eca'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:35.062691 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b697383-6040-453e-8313-8571f3f93a70 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:35.075569 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a80286f9-b251-4cbe-b7d5-75c9db162eca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88db59a73fc14dbbb1d52e252bf1899f', 'request_id': 'req-a80286f9-b251-4cbe-b7d5-75c9db162eca'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:35.075856 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b697383-6040-453e-8313-8571f3f93a70 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:35.128172 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:35.145832 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a80286f9-b251-4cbe-b7d5-75c9db162eca admin admin] The token cache is being invalidate because project 88db59a73fc14dbbb1d52e252bf1899f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:15:35.147148 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a80286f9-b251-4cbe-b7d5-75c9db162eca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:35.149263 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a80286f9-b251-4cbe-b7d5-75c9db162eca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:35.153610 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1010/2047] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:15:34 2022] DELETE /identity/v3/projects/88db59a73fc14dbbb1d52e252bf1899f => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:15:35.162055 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837ebf1-305c-452b-a352-4c6b306f58e7 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:35.162347 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837ebf1-305c-452b-a352-4c6b306f58e7 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:35.162629 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837ebf1-305c-452b-a352-4c6b306f58e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:35.188721 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a08c74-9a56-4b7b-bf55-3991e9716d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:35.190539 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a08c74-9a56-4b7b-bf55-3991e9716d5f admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:35.190896 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a08c74-9a56-4b7b-bf55-3991e9716d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:35.191224 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a08c74-9a56-4b7b-bf55-3991e9716d5f admin admin] PATH_INFO: `/v3/projects/c89b8275371547799668087fdf49eb56/users/721c5a796c38430fb406a242aa053274/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:35.194282 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:35.217353 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a08c74-9a56-4b7b-bf55-3991e9716d5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c89b8275371547799668087fdf49eb56, user_id=721c5a796c38430fb406a242aa053274, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:35.217861 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-9837ebf1-305c-452b-a352-4c6b306f58e7 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:35.219440 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9837ebf1-305c-452b-a352-4c6b306f58e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:35.221318 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a08c74-9a56-4b7b-bf55-3991e9716d5f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:35.250586 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1011/2048] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:35.252612 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61247b0e-7097-48f6-a160-f6fdba8aa1d2 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:35.253098 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61247b0e-7097-48f6-a160-f6fdba8aa1d2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:35.253548 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61247b0e-7097-48f6-a160-f6fdba8aa1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:35.259292 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1038/2049] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:35 2022] PUT /identity/v3/projects/c89b8275371547799668087fdf49eb56/users/721c5a796c38430fb406a242aa053274/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:35.266096 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e61ece4-aa61-46ef-a96e-6498afd4d209 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:35.266509 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e61ece4-aa61-46ef-a96e-6498afd4d209 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:35.266801 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e61ece4-aa61-46ef-a96e-6498afd4d209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:35.288396 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-61247b0e-7097-48f6-a160-f6fdba8aa1d2 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:35.289763 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-61247b0e-7097-48f6-a160-f6fdba8aa1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:35.306368 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5e61ece4-aa61-46ef-a96e-6498afd4d209 None None] MFA Rules not processed for user `721c5a796c38430fb406a242aa053274`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:35.308547 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5e61ece4-aa61-46ef-a96e-6498afd4d209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:35.386746 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1012/2050] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:15:35.391253 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61247b0e-7097-48f6-a160-f6fdba8aa1d2 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:35.404331 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61247b0e-7097-48f6-a160-f6fdba8aa1d2 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:35.410065 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52dac88-a86b-4548-8716-53ed2a8963a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:35.410843 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52dac88-a86b-4548-8716-53ed2a8963a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:35.411247 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52dac88-a86b-4548-8716-53ed2a8963a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:35.411621 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52dac88-a86b-4548-8716-53ed2a8963a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:35.412456 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52dac88-a86b-4548-8716-53ed2a8963a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:15:35.413010 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52dac88-a86b-4548-8716-53ed2a8963a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:35.414822 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52dac88-a86b-4548-8716-53ed2a8963a5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:35.433680 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1013/2051] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:15:35.434243 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1039/2052] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:35.435186 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52dac88-a86b-4548-8716-53ed2a8963a5 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:35.439453 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e61ece4-aa61-46ef-a96e-6498afd4d209 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:35.440847 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5e61ece4-aa61-46ef-a96e-6498afd4d209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:35.450285 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52dac88-a86b-4548-8716-53ed2a8963a5 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:35.457950 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c68c3b-d6b3-4b03-b569-a5df3ca107fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:35.459335 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c68c3b-d6b3-4b03-b569-a5df3ca107fa admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:35.459822 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c68c3b-d6b3-4b03-b569-a5df3ca107fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:35.459950 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e61ece4-aa61-46ef-a96e-6498afd4d209 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:35.460283 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c68c3b-d6b3-4b03-b569-a5df3ca107fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:35.461690 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c68c3b-d6b3-4b03-b569-a5df3ca107fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:35.465752 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c68c3b-d6b3-4b03-b569-a5df3ca107fa admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:35.466121 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14efdbd-88ea-4152-b1ca-ed9bf37f2589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:35.468165 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14efdbd-88ea-4152-b1ca-ed9bf37f2589 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:35.468165 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14efdbd-88ea-4152-b1ca-ed9bf37f2589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:35.468393 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14efdbd-88ea-4152-b1ca-ed9bf37f2589 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:35.469659 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14efdbd-88ea-4152-b1ca-ed9bf37f2589 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:35.472179 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14efdbd-88ea-4152-b1ca-ed9bf37f2589 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:35.514123 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1014/2053] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:15:35 2022] POST /identity/v3/projects => generated 373 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:35.514759 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1040/2054] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:15:35 2022] POST /identity/v3/projects => generated 377 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:35.515863 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14efdbd-88ea-4152-b1ca-ed9bf37f2589 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:35.516375 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c68c3b-d6b3-4b03-b569-a5df3ca107fa admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:35.623869 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14efdbd-88ea-4152-b1ca-ed9bf37f2589 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:35.630564 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae9bf1-5eed-4502-bc7f-5a7283974b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:35.631167 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c68c3b-d6b3-4b03-b569-a5df3ca107fa admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:35.631305 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae9bf1-5eed-4502-bc7f-5a7283974b99 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:35.631601 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae9bf1-5eed-4502-bc7f-5a7283974b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:35.631885 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bae9bf1-5eed-4502-bc7f-5a7283974b99 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:35.632650 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bae9bf1-5eed-4502-bc7f-5a7283974b99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:35.634879 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bae9bf1-5eed-4502-bc7f-5a7283974b99 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:35.639721 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f573c192-c5a4-4f78-81bf-ce86bf76d48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:35.641226 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f573c192-c5a4-4f78-81bf-ce86bf76d48d admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:35.641627 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f573c192-c5a4-4f78-81bf-ce86bf76d48d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:35.641911 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f573c192-c5a4-4f78-81bf-ce86bf76d48d admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:35.642705 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f573c192-c5a4-4f78-81bf-ce86bf76d48d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:35.644786 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f573c192-c5a4-4f78-81bf-ce86bf76d48d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:35.663483 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1041/2055] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:15:35 2022] POST /identity/v3/users => generated 287 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:35.668201 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae9bf1-5eed-4502-bc7f-5a7283974b99 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:35.672934 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1015/2056] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:15:35 2022] POST /identity/v3/users => generated 295 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:35.677848 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f573c192-c5a4-4f78-81bf-ce86bf76d48d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:35.681667 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bae9bf1-5eed-4502-bc7f-5a7283974b99 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:35.689547 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f573c192-c5a4-4f78-81bf-ce86bf76d48d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:35.692819 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250a3470-b67a-40af-a896-dd9f8d2ce952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:35.693895 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a3470-b67a-40af-a896-dd9f8d2ce952 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:35.694362 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a3470-b67a-40af-a896-dd9f8d2ce952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:35.694895 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a3470-b67a-40af-a896-dd9f8d2ce952 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:35.695979 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250a3470-b67a-40af-a896-dd9f8d2ce952 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:35.698431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250a3470-b67a-40af-a896-dd9f8d2ce952 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:35.699017 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78bf117-5b4b-4084-bfe9-e6164e0ed9ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:35.699929 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78bf117-5b4b-4084-bfe9-e6164e0ed9ec admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:35.700292 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78bf117-5b4b-4084-bfe9-e6164e0ed9ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:35.700642 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78bf117-5b4b-4084-bfe9-e6164e0ed9ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:35.704830 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78bf117-5b4b-4084-bfe9-e6164e0ed9ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:35.706836 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78bf117-5b4b-4084-bfe9-e6164e0ed9ec admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:35.715500 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1042/2057] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:35.719388 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250a3470-b67a-40af-a896-dd9f8d2ce952 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:35.722325 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1016/2058] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:35.726170 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78bf117-5b4b-4084-bfe9-e6164e0ed9ec admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:35.735003 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250a3470-b67a-40af-a896-dd9f8d2ce952 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:35.746064 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b9e078-e9cc-4e72-a404-c4669aab086b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:35.746399 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78bf117-5b4b-4084-bfe9-e6164e0ed9ec admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:35.746820 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9e078-e9cc-4e72-a404-c4669aab086b admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:35.747146 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9e078-e9cc-4e72-a404-c4669aab086b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:35.747419 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9e078-e9cc-4e72-a404-c4669aab086b admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:35.748148 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b9e078-e9cc-4e72-a404-c4669aab086b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:35.749777 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b9e078-e9cc-4e72-a404-c4669aab086b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:35.752575 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c2631a-bb41-4f09-bf44-7470933a5d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:35.753992 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2631a-bb41-4f09-bf44-7470933a5d3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:35.754565 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2631a-bb41-4f09-bf44-7470933a5d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:35.754931 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c2631a-bb41-4f09-bf44-7470933a5d3e admin admin] PATH_INFO: `/v3/projects/ec9ecb18f3484a20ae4eb44472a338dd/users/33ba679717874332b0364471cfded983/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:35.766634 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1043/2059] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:15:35.771181 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b9e078-e9cc-4e72-a404-c4669aab086b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:35.774925 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c2631a-bb41-4f09-bf44-7470933a5d3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec9ecb18f3484a20ae4eb44472a338dd, user_id=33ba679717874332b0364471cfded983, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:35.778620 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c2631a-bb41-4f09-bf44-7470933a5d3e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:35.782710 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b9e078-e9cc-4e72-a404-c4669aab086b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:35.788310 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e9343-bce4-45e2-b650-a960c46bc8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:35.789704 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e9343-bce4-45e2-b650-a960c46bc8e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:35.790199 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e9343-bce4-45e2-b650-a960c46bc8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:35.790497 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e9343-bce4-45e2-b650-a960c46bc8e6 admin admin] PATH_INFO: `/v3/projects/5ae54ed81d14436d826655396e5fe6ab/users/206a442fe7de4765869b39aa997b197c/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:35.804331 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1017/2060] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:35 2022] PUT /identity/v3/projects/ec9ecb18f3484a20ae4eb44472a338dd/users/33ba679717874332b0364471cfded983/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:15:35.805611 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c2631a-bb41-4f09-bf44-7470933a5d3e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:35.815514 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180e9343-bce4-45e2-b650-a960c46bc8e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ae54ed81d14436d826655396e5fe6ab, user_id=206a442fe7de4765869b39aa997b197c, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:35.818281 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180e9343-bce4-45e2-b650-a960c46bc8e6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:35.821354 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:35.833828 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:35.839385 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1044/2061] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:35 2022] PUT /identity/v3/projects/5ae54ed81d14436d826655396e5fe6ab/users/206a442fe7de4765869b39aa997b197c/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:15:35.860468 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85713863-b266-4449-bd3b-533fcb0dd7f2 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:35.860721 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85713863-b266-4449-bd3b-533fcb0dd7f2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:35.861013 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85713863-b266-4449-bd3b-533fcb0dd7f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:35.906257 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-85713863-b266-4449-bd3b-533fcb0dd7f2 None None] MFA Rules not processed for user `206a442fe7de4765869b39aa997b197c`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:35.907645 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-85713863-b266-4449-bd3b-533fcb0dd7f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:35.941048 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c2631a-bb41-4f09-bf44-7470933a5d3e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:36.006619 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1045/2062] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:36.018597 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85713863-b266-4449-bd3b-533fcb0dd7f2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:36.019989 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-85713863-b266-4449-bd3b-533fcb0dd7f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:36.028105 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dd135d-8684-4374-821d-1fa7046527a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:36.032266 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dd135d-8684-4374-821d-1fa7046527a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:36.032266 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dd135d-8684-4374-821d-1fa7046527a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:36.032266 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dd135d-8684-4374-821d-1fa7046527a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:36.032266 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dd135d-8684-4374-821d-1fa7046527a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:36.033590 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dd135d-8684-4374-821d-1fa7046527a8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:36.046739 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1018/2063] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:36.051042 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dd135d-8684-4374-821d-1fa7046527a8 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:36.061663 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dd135d-8684-4374-821d-1fa7046527a8 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:36.066298 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5cc685-67fb-48d0-925d-7fcc8720b29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:36.067215 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5cc685-67fb-48d0-925d-7fcc8720b29f admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:36.067499 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5cc685-67fb-48d0-925d-7fcc8720b29f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:36.067776 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5cc685-67fb-48d0-925d-7fcc8720b29f admin admin] PATH_INFO: `/v3/projects/ec9ecb18f3484a20ae4eb44472a338dd/users/33ba679717874332b0364471cfded983/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:36.069754 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5cc685-67fb-48d0-925d-7fcc8720b29f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec9ecb18f3484a20ae4eb44472a338dd, user_id=33ba679717874332b0364471cfded983, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:36.073311 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5cc685-67fb-48d0-925d-7fcc8720b29f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:36.086741 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1019/2064] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:36 2022] PUT /identity/v3/projects/ec9ecb18f3484a20ae4eb44472a338dd/users/33ba679717874332b0364471cfded983/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:15:36.094078 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85713863-b266-4449-bd3b-533fcb0dd7f2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:36.098503 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27188884-64f5-4f0a-93df-a3fd8a65ebb1 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:36.098794 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27188884-64f5-4f0a-93df-a3fd8a65ebb1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:36.099106 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27188884-64f5-4f0a-93df-a3fd8a65ebb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:36.135146 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-27188884-64f5-4f0a-93df-a3fd8a65ebb1 None None] MFA Rules not processed for user `33ba679717874332b0364471cfded983`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:36.137119 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-27188884-64f5-4f0a-93df-a3fd8a65ebb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:36.172415 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8044138e-141a-4219-b5b3-738080a30b68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:36.173256 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044138e-141a-4219-b5b3-738080a30b68 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:36.173761 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044138e-141a-4219-b5b3-738080a30b68 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:36.174243 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044138e-141a-4219-b5b3-738080a30b68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:36.236948 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1020/2065] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:36 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:15:36.242993 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:36.247599 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8044138e-141a-4219-b5b3-738080a30b68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:36.249764 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8044138e-141a-4219-b5b3-738080a30b68 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:36.259748 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1046/2066] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:36 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:15:36.293074 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27188884-64f5-4f0a-93df-a3fd8a65ebb1 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:36.293074 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-27188884-64f5-4f0a-93df-a3fd8a65ebb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:36.302921 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27188884-64f5-4f0a-93df-a3fd8a65ebb1 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:36.306881 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ffd5cf-21aa-4e89-8e77-da50abe6e368 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:36.307562 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffd5cf-21aa-4e89-8e77-da50abe6e368 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:36.307847 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffd5cf-21aa-4e89-8e77-da50abe6e368 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:36.308128 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ffd5cf-21aa-4e89-8e77-da50abe6e368 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:36.309954 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f1ffd5cf-21aa-4e89-8e77-da50abe6e368 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:36.339133 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:36.374342 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ffd5cf-21aa-4e89-8e77-da50abe6e368 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:36.375714 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ffd5cf-21aa-4e89-8e77-da50abe6e368 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:36.385011 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1021/2067] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:38.103297 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8044138e-141a-4219-b5b3-738080a30b68 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:38.112598 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8044138e-141a-4219-b5b3-738080a30b68 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:38.118355 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38df51f2-ef70-4926-a95c-4b93268be7e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:38.119252 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38df51f2-ef70-4926-a95c-4b93268be7e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:38.119739 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38df51f2-ef70-4926-a95c-4b93268be7e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:38.120166 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38df51f2-ef70-4926-a95c-4b93268be7e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:38.207117 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38df51f2-ef70-4926-a95c-4b93268be7e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:38.208753 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38df51f2-ef70-4926-a95c-4b93268be7e3 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:38.219682 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1047/2068] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:38 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:40.550878 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ffd5cf-21aa-4e89-8e77-da50abe6e368 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:40.560092 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ffd5cf-21aa-4e89-8e77-da50abe6e368 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:40.563903 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b854a4-d1bd-4774-8cbc-2a6f0577048f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:40.564556 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b854a4-d1bd-4774-8cbc-2a6f0577048f service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:40.564849 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b854a4-d1bd-4774-8cbc-2a6f0577048f service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:40.565246 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b854a4-d1bd-4774-8cbc-2a6f0577048f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:40.576340 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b854a4-d1bd-4774-8cbc-2a6f0577048f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:40.577847 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b854a4-d1bd-4774-8cbc-2a6f0577048f service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:40.588137 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1022/2069] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:40 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:41.558053 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38df51f2-ef70-4926-a95c-4b93268be7e3 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:41.653823 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38df51f2-ef70-4926-a95c-4b93268be7e3 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:41.658711 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f76f7cd-9596-445e-b0de-a3e2bcf3ddba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:41.659754 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76f7cd-9596-445e-b0de-a3e2bcf3ddba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:41.660108 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76f7cd-9596-445e-b0de-a3e2bcf3ddba admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:41.660412 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f76f7cd-9596-445e-b0de-a3e2bcf3ddba admin admin] PATH_INFO: `/v3/users/206a442fe7de4765869b39aa997b197c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:41.662700 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f76f7cd-9596-445e-b0de-a3e2bcf3ddba admin admin] RBAC: Authorizing `identity:delete_user(user_id=206a442fe7de4765869b39aa997b197c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:41.665894 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f76f7cd-9596-445e-b0de-a3e2bcf3ddba admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:41.787610 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8f76f7cd-9596-445e-b0de-a3e2bcf3ddba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '206a442fe7de4765869b39aa997b197c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:41.820997 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8f76f7cd-9596-445e-b0de-a3e2bcf3ddba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '206a442fe7de4765869b39aa997b197c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:41.823363 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8f76f7cd-9596-445e-b0de-a3e2bcf3ddba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '206a442fe7de4765869b39aa997b197c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:41.844500 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8f76f7cd-9596-445e-b0de-a3e2bcf3ddba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '206a442fe7de4765869b39aa997b197c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:41.845304 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8f76f7cd-9596-445e-b0de-a3e2bcf3ddba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '206a442fe7de4765869b39aa997b197c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:41.910472 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1048/2070] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:15:41 2022] DELETE /identity/v3/users/206a442fe7de4765869b39aa997b197c => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:41.915107 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7809663a-4c1a-4cf0-ac6c-cddb8ccbcd6c None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:41.915609 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7809663a-4c1a-4cf0-ac6c-cddb8ccbcd6c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:41.916871 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7809663a-4c1a-4cf0-ac6c-cddb8ccbcd6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:41.969781 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7809663a-4c1a-4cf0-ac6c-cddb8ccbcd6c None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:41.972990 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7809663a-4c1a-4cf0-ac6c-cddb8ccbcd6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:42.052495 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1023/2071] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:42.069155 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f76f7cd-9596-445e-b0de-a3e2bcf3ddba admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:42.070467 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8f76f7cd-9596-445e-b0de-a3e2bcf3ddba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:42.100181 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:42.175441 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f76f7cd-9596-445e-b0de-a3e2bcf3ddba admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:42.180141 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f02dc54-a5b4-4db9-8f2f-e73968e2de5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:42.180830 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02dc54-a5b4-4db9-8f2f-e73968e2de5f service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:42.181159 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02dc54-a5b4-4db9-8f2f-e73968e2de5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:42.181459 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02dc54-a5b4-4db9-8f2f-e73968e2de5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:42.193764 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f02dc54-a5b4-4db9-8f2f-e73968e2de5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:42.195627 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f02dc54-a5b4-4db9-8f2f-e73968e2de5f service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:42.206788 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1049/2072] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:42.571005 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7809663a-4c1a-4cf0-ac6c-cddb8ccbcd6c None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:42.572460 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7809663a-4c1a-4cf0-ac6c-cddb8ccbcd6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:42.587705 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7809663a-4c1a-4cf0-ac6c-cddb8ccbcd6c None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:42.595086 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518c37ef-1138-43f6-aeba-33c00b7b1d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:42.596304 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c37ef-1138-43f6-aeba-33c00b7b1d6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:42.596772 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c37ef-1138-43f6-aeba-33c00b7b1d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:42.597286 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c37ef-1138-43f6-aeba-33c00b7b1d6a admin admin] PATH_INFO: `/v3/projects/5ae54ed81d14436d826655396e5fe6ab` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:42.608807 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518c37ef-1138-43f6-aeba-33c00b7b1d6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ae54ed81d14436d826655396e5fe6ab)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:42.610783 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518c37ef-1138-43f6-aeba-33c00b7b1d6a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:42.657884 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-518c37ef-1138-43f6-aeba-33c00b7b1d6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ae54ed81d14436d826655396e5fe6ab', 'request_id': 'req-518c37ef-1138-43f6-aeba-33c00b7b1d6a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:42.664239 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-518c37ef-1138-43f6-aeba-33c00b7b1d6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ae54ed81d14436d826655396e5fe6ab', 'request_id': 'req-518c37ef-1138-43f6-aeba-33c00b7b1d6a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:42.670919 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-518c37ef-1138-43f6-aeba-33c00b7b1d6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ae54ed81d14436d826655396e5fe6ab', 'request_id': 'req-518c37ef-1138-43f6-aeba-33c00b7b1d6a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:42.683125 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-518c37ef-1138-43f6-aeba-33c00b7b1d6a admin admin] The token cache is being invalidate because project 5ae54ed81d14436d826655396e5fe6ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:15:42.683407 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-518c37ef-1138-43f6-aeba-33c00b7b1d6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:42.684029 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-518c37ef-1138-43f6-aeba-33c00b7b1d6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:42.685582 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1024/2073] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:15:42 2022] DELETE /identity/v3/projects/5ae54ed81d14436d826655396e5fe6ab => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:42.691029 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04889a9-1bb3-4f07-ae8d-8b216079191b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:42.691562 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04889a9-1bb3-4f07-ae8d-8b216079191b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:42.691677 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04889a9-1bb3-4f07-ae8d-8b216079191b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:42.710010 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d04889a9-1bb3-4f07-ae8d-8b216079191b None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:42.711148 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d04889a9-1bb3-4f07-ae8d-8b216079191b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:42.781387 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1050/2074] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:15:42.790992 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef313d7-2790-4d7f-b6e3-7a87cd31c2b4 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:42.791315 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef313d7-2790-4d7f-b6e3-7a87cd31c2b4 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:42.791600 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef313d7-2790-4d7f-b6e3-7a87cd31c2b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:42.812096 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-bef313d7-2790-4d7f-b6e3-7a87cd31c2b4 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:42.813504 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bef313d7-2790-4d7f-b6e3-7a87cd31c2b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:42.821860 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1025/2075] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:42.826687 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04889a9-1bb3-4f07-ae8d-8b216079191b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:42.836902 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04889a9-1bb3-4f07-ae8d-8b216079191b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:42.843227 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08548749-2598-4fc2-afd8-267bc0a604f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:42.843921 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08548749-2598-4fc2-afd8-267bc0a604f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:42.844234 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08548749-2598-4fc2-afd8-267bc0a604f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:42.844498 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08548749-2598-4fc2-afd8-267bc0a604f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:42.845349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08548749-2598-4fc2-afd8-267bc0a604f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:15:42.845779 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08548749-2598-4fc2-afd8-267bc0a604f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:42.847599 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08548749-2598-4fc2-afd8-267bc0a604f5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:42.866581 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1051/2076] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:15:42.867798 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef313d7-2790-4d7f-b6e3-7a87cd31c2b4 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:42.879153 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef313d7-2790-4d7f-b6e3-7a87cd31c2b4 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:42.884000 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706d9a1f-b29b-4079-8dc9-adbae46688ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:42.884656 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d9a1f-b29b-4079-8dc9-adbae46688ad admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:42.884968 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d9a1f-b29b-4079-8dc9-adbae46688ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:42.885818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706d9a1f-b29b-4079-8dc9-adbae46688ad admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:42.886528 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706d9a1f-b29b-4079-8dc9-adbae46688ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:42.888621 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706d9a1f-b29b-4079-8dc9-adbae46688ad admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:42.918198 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1026/2077] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:15:42 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:42.922507 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08548749-2598-4fc2-afd8-267bc0a604f5 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:42.935665 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08548749-2598-4fc2-afd8-267bc0a604f5 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:43.000835 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d64e13-1773-49fa-8fbd-6e39e909399a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:43.001892 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d64e13-1773-49fa-8fbd-6e39e909399a admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:43.002176 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d64e13-1773-49fa-8fbd-6e39e909399a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:43.002452 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d64e13-1773-49fa-8fbd-6e39e909399a admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:43.003318 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d64e13-1773-49fa-8fbd-6e39e909399a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:43.005436 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d64e13-1773-49fa-8fbd-6e39e909399a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:43.027912 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1052/2078] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:15:42 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:43.032169 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706d9a1f-b29b-4079-8dc9-adbae46688ad admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:43.042468 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706d9a1f-b29b-4079-8dc9-adbae46688ad admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:43.047085 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1a28b2-ec82-493c-935c-299bb4a2db75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:43.047740 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1a28b2-ec82-493c-935c-299bb4a2db75 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:43.048024 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1a28b2-ec82-493c-935c-299bb4a2db75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:43.048300 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1a28b2-ec82-493c-935c-299bb4a2db75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:43.049026 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1a28b2-ec82-493c-935c-299bb4a2db75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:43.050621 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1a28b2-ec82-493c-935c-299bb4a2db75 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:43.052661 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:43.065947 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1027/2079] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:43.067105 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d64e13-1773-49fa-8fbd-6e39e909399a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:43.080566 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d64e13-1773-49fa-8fbd-6e39e909399a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:43.085277 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e01f714-e30a-45aa-a206-97bff349cdc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:43.085940 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01f714-e30a-45aa-a206-97bff349cdc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:43.086260 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01f714-e30a-45aa-a206-97bff349cdc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:43.086535 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01f714-e30a-45aa-a206-97bff349cdc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:43.087301 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e01f714-e30a-45aa-a206-97bff349cdc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:43.088894 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e01f714-e30a-45aa-a206-97bff349cdc1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:43.099319 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1053/2080] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:43.103463 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1a28b2-ec82-493c-935c-299bb4a2db75 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:43.113605 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1a28b2-ec82-493c-935c-299bb4a2db75 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:43.118569 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b43c06-42ed-491f-8402-d659b9eeb308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:43.120008 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b43c06-42ed-491f-8402-d659b9eeb308 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:43.120296 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b43c06-42ed-491f-8402-d659b9eeb308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:43.120720 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b43c06-42ed-491f-8402-d659b9eeb308 admin admin] PATH_INFO: `/v3/projects/a5a48e1060474f21aacf525d26371675/users/34ac2a8ad2e343838f8e3e3285ec5f55/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:43.138363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b43c06-42ed-491f-8402-d659b9eeb308 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5a48e1060474f21aacf525d26371675, user_id=34ac2a8ad2e343838f8e3e3285ec5f55, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:43.141063 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b43c06-42ed-491f-8402-d659b9eeb308 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:43.157679 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1028/2081] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:43 2022] PUT /identity/v3/projects/a5a48e1060474f21aacf525d26371675/users/34ac2a8ad2e343838f8e3e3285ec5f55/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:43.163715 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053b627-5727-4b98-b9e7-444a4892dd17 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:43.164054 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053b627-5727-4b98-b9e7-444a4892dd17 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:43.164410 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6053b627-5727-4b98-b9e7-444a4892dd17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:43.192646 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6053b627-5727-4b98-b9e7-444a4892dd17 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:43.194087 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6053b627-5727-4b98-b9e7-444a4892dd17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:43.217598 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:43.272508 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1054/2082] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:43.284096 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b43c06-42ed-491f-8402-d659b9eeb308 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:43.286281 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f6b43c06-42ed-491f-8402-d659b9eeb308 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:43.372826 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b43c06-42ed-491f-8402-d659b9eeb308 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:43.377619 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fcb1ff-20dc-4e8b-9a6a-fc48b0d465d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:43.378455 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fcb1ff-20dc-4e8b-9a6a-fc48b0d465d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:43.378750 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fcb1ff-20dc-4e8b-9a6a-fc48b0d465d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:43.379215 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fcb1ff-20dc-4e8b-9a6a-fc48b0d465d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:43.391503 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fcb1ff-20dc-4e8b-9a6a-fc48b0d465d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:43.392943 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fcb1ff-20dc-4e8b-9a6a-fc48b0d465d2 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:43.404219 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1029/2083] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:43.443108 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6053b627-5727-4b98-b9e7-444a4892dd17 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:43.445063 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6053b627-5727-4b98-b9e7-444a4892dd17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:43.463801 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6053b627-5727-4b98-b9e7-444a4892dd17 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:43.472586 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b8591-a5af-4959-be38-23c93fd07e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:43.473485 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b8591-a5af-4959-be38-23c93fd07e13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:43.473773 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b8591-a5af-4959-be38-23c93fd07e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:43.474144 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b8591-a5af-4959-be38-23c93fd07e13 admin admin] PATH_INFO: `/v3/users/b72ccd82e12c477eb9c4ea151b32ba86` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:43.481268 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124b8591-a5af-4959-be38-23c93fd07e13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b72ccd82e12c477eb9c4ea151b32ba86)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:43.484827 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124b8591-a5af-4959-be38-23c93fd07e13 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:43.554426 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-124b8591-a5af-4959-be38-23c93fd07e13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b72ccd82e12c477eb9c4ea151b32ba86'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:43.563294 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-124b8591-a5af-4959-be38-23c93fd07e13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b72ccd82e12c477eb9c4ea151b32ba86'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:43.564325 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-124b8591-a5af-4959-be38-23c93fd07e13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b72ccd82e12c477eb9c4ea151b32ba86'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:43.575171 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-124b8591-a5af-4959-be38-23c93fd07e13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b72ccd82e12c477eb9c4ea151b32ba86'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:43.576210 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-124b8591-a5af-4959-be38-23c93fd07e13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b72ccd82e12c477eb9c4ea151b32ba86'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:43.611851 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1055/2084] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:15:43 2022] DELETE /identity/v3/users/b72ccd82e12c477eb9c4ea151b32ba86 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:43.616634 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e8897-550b-4f2c-bf1c-c1f001ff7402 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:43.616855 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e8897-550b-4f2c-bf1c-c1f001ff7402 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:43.617085 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e8897-550b-4f2c-bf1c-c1f001ff7402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:43.669319 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-fd3e8897-550b-4f2c-bf1c-c1f001ff7402 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:43.670946 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fd3e8897-550b-4f2c-bf1c-c1f001ff7402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:43.757501 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1030/2085] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:15:43.771733 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b8591-a5af-4959-be38-23c93fd07e13 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:43.774289 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-124b8591-a5af-4959-be38-23c93fd07e13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:43.882303 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b8591-a5af-4959-be38-23c93fd07e13 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:43.888998 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1875bfa5-087f-4724-977f-22ac016ebb4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:43.891013 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875bfa5-087f-4724-977f-22ac016ebb4a service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:43.891458 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875bfa5-087f-4724-977f-22ac016ebb4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:43.892261 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875bfa5-087f-4724-977f-22ac016ebb4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:43.907680 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1875bfa5-087f-4724-977f-22ac016ebb4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:43.909872 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1875bfa5-087f-4724-977f-22ac016ebb4a service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:43.926755 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1056/2086] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:44.316592 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42faa3-03cb-4d40-8bf7-1c256530ad63 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:44.317183 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42faa3-03cb-4d40-8bf7-1c256530ad63 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:44.317183 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e42faa3-03cb-4d40-8bf7-1c256530ad63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:44.353179 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1875bfa5-087f-4724-977f-22ac016ebb4a service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:44.355331 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1875bfa5-087f-4724-977f-22ac016ebb4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:44.373720 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5e42faa3-03cb-4d40-8bf7-1c256530ad63 None None] MFA Rules not processed for user `34ac2a8ad2e343838f8e3e3285ec5f55`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:44.376181 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5e42faa3-03cb-4d40-8bf7-1c256530ad63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:44.383672 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1875bfa5-087f-4724-977f-22ac016ebb4a service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:44.396194 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f0cdd-bf94-4049-8ef9-c8a27f536e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:44.397349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f0cdd-bf94-4049-8ef9-c8a27f536e56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:44.397784 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f0cdd-bf94-4049-8ef9-c8a27f536e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:44.399013 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f0cdd-bf94-4049-8ef9-c8a27f536e56 admin admin] PATH_INFO: `/v3/projects/1159939967494e57aabf510ed7c353ce` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:44.400543 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f0cdd-bf94-4049-8ef9-c8a27f536e56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1159939967494e57aabf510ed7c353ce)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:44.405026 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f0cdd-bf94-4049-8ef9-c8a27f536e56 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:44.460082 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:44.478561 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0f0f0cdd-bf94-4049-8ef9-c8a27f536e56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1159939967494e57aabf510ed7c353ce', 'request_id': 'req-0f0f0cdd-bf94-4049-8ef9-c8a27f536e56'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:44.489813 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0f0f0cdd-bf94-4049-8ef9-c8a27f536e56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1159939967494e57aabf510ed7c353ce', 'request_id': 'req-0f0f0cdd-bf94-4049-8ef9-c8a27f536e56'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:44.497215 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1031/2087] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:44.499248 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0f0f0cdd-bf94-4049-8ef9-c8a27f536e56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1159939967494e57aabf510ed7c353ce', 'request_id': 'req-0f0f0cdd-bf94-4049-8ef9-c8a27f536e56'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:44.512961 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0f0f0cdd-bf94-4049-8ef9-c8a27f536e56 admin admin] The token cache is being invalidate because project 1159939967494e57aabf510ed7c353ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:15:44.513325 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0f0f0cdd-bf94-4049-8ef9-c8a27f536e56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:44.514116 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0f0f0cdd-bf94-4049-8ef9-c8a27f536e56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:44.516203 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1057/2088] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:15:44 2022] DELETE /identity/v3/projects/1159939967494e57aabf510ed7c353ce => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:44.517198 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42faa3-03cb-4d40-8bf7-1c256530ad63 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:44.518775 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5e42faa3-03cb-4d40-8bf7-1c256530ad63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:44.520844 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025d45b4-a01e-4f3e-ab56-ce229870ce02 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:44.521202 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025d45b4-a01e-4f3e-ab56-ce229870ce02 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:44.521551 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025d45b4-a01e-4f3e-ab56-ce229870ce02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:44.533076 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:44.546088 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-025d45b4-a01e-4f3e-ab56-ce229870ce02 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:44.547599 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-025d45b4-a01e-4f3e-ab56-ce229870ce02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:44.593309 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e42faa3-03cb-4d40-8bf7-1c256530ad63 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:44.612927 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1058/2089] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:44.620278 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a6408-3e37-4411-8965-d6ea860c4efd None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:44.620565 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a6408-3e37-4411-8965-d6ea860c4efd None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:44.620862 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a6408-3e37-4411-8965-d6ea860c4efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:44.639693 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d51a6408-3e37-4411-8965-d6ea860c4efd None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:44.640987 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d51a6408-3e37-4411-8965-d6ea860c4efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:44.645460 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47efa8ab-2a11-4b4b-bd95-51e59666302e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:44.646144 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47efa8ab-2a11-4b4b-bd95-51e59666302e service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:44.646427 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47efa8ab-2a11-4b4b-bd95-51e59666302e service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:44.646737 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47efa8ab-2a11-4b4b-bd95-51e59666302e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:44.648600 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-47efa8ab-2a11-4b4b-bd95-51e59666302e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:44.649298 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1059/2090] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:15:44.653965 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a6408-3e37-4411-8965-d6ea860c4efd None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:44.663920 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a6408-3e37-4411-8965-d6ea860c4efd None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:44.669147 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd53e34-a8ce-4cf9-b9a2-5d2d27759f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:44.669792 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd53e34-a8ce-4cf9-b9a2-5d2d27759f59 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:44.670078 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd53e34-a8ce-4cf9-b9a2-5d2d27759f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:44.670357 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd53e34-a8ce-4cf9-b9a2-5d2d27759f59 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:44.671117 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd53e34-a8ce-4cf9-b9a2-5d2d27759f59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:15:44.671423 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd53e34-a8ce-4cf9-b9a2-5d2d27759f59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:44.673034 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd53e34-a8ce-4cf9-b9a2-5d2d27759f59 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:44.684675 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1060/2091] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:15:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:15:44.689098 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd53e34-a8ce-4cf9-b9a2-5d2d27759f59 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:44.699000 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd53e34-a8ce-4cf9-b9a2-5d2d27759f59 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:44.703784 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a23f7bd-fbe0-49c7-b028-ee365bcb69b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:44.704222 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a23f7bd-fbe0-49c7-b028-ee365bcb69b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:44.704504 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a23f7bd-fbe0-49c7-b028-ee365bcb69b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:44.704776 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a23f7bd-fbe0-49c7-b028-ee365bcb69b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:44.705457 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a23f7bd-fbe0-49c7-b028-ee365bcb69b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:44.707357 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a23f7bd-fbe0-49c7-b028-ee365bcb69b6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:44.715042 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47efa8ab-2a11-4b4b-bd95-51e59666302e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:44.716558 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47efa8ab-2a11-4b4b-bd95-51e59666302e service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:44.727804 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1032/2092] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:44.736464 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1061/2093] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:15:44 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:15:44.740618 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47efa8ab-2a11-4b4b-bd95-51e59666302e service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:44.768543 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47efa8ab-2a11-4b4b-bd95-51e59666302e service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:44.817575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a23f7bd-fbe0-49c7-b028-ee365bcb69b6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:44.820405 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3a23f7bd-fbe0-49c7-b028-ee365bcb69b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:44.839672 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f9f316-6787-4f7d-9b13-483c69aef020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:44.840527 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f9f316-6787-4f7d-9b13-483c69aef020 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:44.840821 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f9f316-6787-4f7d-9b13-483c69aef020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:44.841106 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f9f316-6787-4f7d-9b13-483c69aef020 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:44.841802 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f9f316-6787-4f7d-9b13-483c69aef020 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:44.844040 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f9f316-6787-4f7d-9b13-483c69aef020 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:44.870889 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1033/2094] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:15:44 2022] POST /identity/v3/users => generated 292 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:44.876506 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f9f316-6787-4f7d-9b13-483c69aef020 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:44.892584 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f9f316-6787-4f7d-9b13-483c69aef020 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:44.898381 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50e38d4-4b9d-4b60-bfda-48807bbb8db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:44.899524 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50e38d4-4b9d-4b60-bfda-48807bbb8db6 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:44.899893 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50e38d4-4b9d-4b60-bfda-48807bbb8db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:44.900238 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50e38d4-4b9d-4b60-bfda-48807bbb8db6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:44.901568 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50e38d4-4b9d-4b60-bfda-48807bbb8db6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:44.903620 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50e38d4-4b9d-4b60-bfda-48807bbb8db6 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:44.905233 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:44.905475 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a23f7bd-fbe0-49c7-b028-ee365bcb69b6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:44.914451 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8986432f-c88d-4e0e-a2ec-5e0a62646f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:44.919773 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8986432f-c88d-4e0e-a2ec-5e0a62646f29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:44.919773 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8986432f-c88d-4e0e-a2ec-5e0a62646f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:44.920065 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8986432f-c88d-4e0e-a2ec-5e0a62646f29 admin admin] PATH_INFO: `/v3/users/4e9116d5ca37444983ed48243f4828ad` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:44.921503 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8986432f-c88d-4e0e-a2ec-5e0a62646f29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e9116d5ca37444983ed48243f4828ad)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:44.923036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1034/2095] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:44 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:44.923626 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8986432f-c88d-4e0e-a2ec-5e0a62646f29 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:44.924288 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50e38d4-4b9d-4b60-bfda-48807bbb8db6 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:44.937377 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50e38d4-4b9d-4b60-bfda-48807bbb8db6 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:44.943299 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3620f13e-e9ba-49be-86ac-f44a9751d6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:44.943942 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3620f13e-e9ba-49be-86ac-f44a9751d6ef admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:44.944304 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3620f13e-e9ba-49be-86ac-f44a9751d6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:44.944570 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3620f13e-e9ba-49be-86ac-f44a9751d6ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:44.945319 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3620f13e-e9ba-49be-86ac-f44a9751d6ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:44.947076 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3620f13e-e9ba-49be-86ac-f44a9751d6ef admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:44.952995 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:44.958798 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1035/2096] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:44.963492 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3620f13e-e9ba-49be-86ac-f44a9751d6ef admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:44.976047 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3620f13e-e9ba-49be-86ac-f44a9751d6ef admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:44.983095 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466d04b0-fab6-4787-8842-e15dc9f69bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:44.984007 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d04b0-fab6-4787-8842-e15dc9f69bc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:44.984287 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d04b0-fab6-4787-8842-e15dc9f69bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:44.984563 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466d04b0-fab6-4787-8842-e15dc9f69bc0 admin admin] PATH_INFO: `/v3/projects/7facd29ad0d746099537a1b1345c2c90/users/50ca9d699b004df4becfe3241802fb17/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:44.996383 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8986432f-c88d-4e0e-a2ec-5e0a62646f29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e9116d5ca37444983ed48243f4828ad'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:45.002596 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466d04b0-fab6-4787-8842-e15dc9f69bc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7facd29ad0d746099537a1b1345c2c90, user_id=50ca9d699b004df4becfe3241802fb17, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:45.004559 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8986432f-c88d-4e0e-a2ec-5e0a62646f29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e9116d5ca37444983ed48243f4828ad'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:45.005564 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8986432f-c88d-4e0e-a2ec-5e0a62646f29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e9116d5ca37444983ed48243f4828ad'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:45.005684 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466d04b0-fab6-4787-8842-e15dc9f69bc0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:45.014646 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8986432f-c88d-4e0e-a2ec-5e0a62646f29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e9116d5ca37444983ed48243f4828ad'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:45.015522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8986432f-c88d-4e0e-a2ec-5e0a62646f29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e9116d5ca37444983ed48243f4828ad'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:45.021068 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1036/2097] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:44 2022] PUT /identity/v3/projects/7facd29ad0d746099537a1b1345c2c90/users/50ca9d699b004df4becfe3241802fb17/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:45.025841 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d98e4b8-a313-4527-b7f9-005ede142079 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:45.026270 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d98e4b8-a313-4527-b7f9-005ede142079 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:45.026769 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d98e4b8-a313-4527-b7f9-005ede142079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:45.039294 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1062/2098] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:15:44 2022] DELETE /identity/v3/users/4e9116d5ca37444983ed48243f4828ad => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:15:45.053149 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414afe23-8baa-45b0-bcf6-cbaa33431574 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:45.053149 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414afe23-8baa-45b0-bcf6-cbaa33431574 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:45.053149 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414afe23-8baa-45b0-bcf6-cbaa33431574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:45.064201 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-6d98e4b8-a313-4527-b7f9-005ede142079 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:45.065471 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6d98e4b8-a313-4527-b7f9-005ede142079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:45.074024 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-414afe23-8baa-45b0-bcf6-cbaa33431574 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:45.076753 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-414afe23-8baa-45b0-bcf6-cbaa33431574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:45.155959 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1037/2099] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:45.165759 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d98e4b8-a313-4527-b7f9-005ede142079 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:45.167418 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6d98e4b8-a313-4527-b7f9-005ede142079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:45.181224 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1063/2100] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:45.192974 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414afe23-8baa-45b0-bcf6-cbaa33431574 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:45.194377 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-414afe23-8baa-45b0-bcf6-cbaa33431574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:45.255915 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d98e4b8-a313-4527-b7f9-005ede142079 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:45.261372 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11846afa-9386-4806-82f7-3c602ea09573 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:45.262510 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11846afa-9386-4806-82f7-3c602ea09573 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:45.262510 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11846afa-9386-4806-82f7-3c602ea09573 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:45.262510 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11846afa-9386-4806-82f7-3c602ea09573 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:45.268867 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414afe23-8baa-45b0-bcf6-cbaa33431574 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:45.273038 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09ee753-577c-43a5-b82c-09acf0b62b1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:45.273513 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11846afa-9386-4806-82f7-3c602ea09573 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:45.273861 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ee753-577c-43a5-b82c-09acf0b62b1a service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:45.274201 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ee753-577c-43a5-b82c-09acf0b62b1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:45.274478 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ee753-577c-43a5-b82c-09acf0b62b1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:45.275101 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11846afa-9386-4806-82f7-3c602ea09573 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:45.284889 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09ee753-577c-43a5-b82c-09acf0b62b1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:45.286545 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09ee753-577c-43a5-b82c-09acf0b62b1a service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:45.286697 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1038/2101] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:45.297165 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1064/2102] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:15:45.823210 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11846afa-9386-4806-82f7-3c602ea09573 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:45.824944 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-11846afa-9386-4806-82f7-3c602ea09573 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:45.849663 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11846afa-9386-4806-82f7-3c602ea09573 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:45.864288 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb9062-ae53-4a50-8082-eba7b97c95ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:45.865675 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb9062-ae53-4a50-8082-eba7b97c95ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:45.868064 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb9062-ae53-4a50-8082-eba7b97c95ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:45.868521 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb9062-ae53-4a50-8082-eba7b97c95ec admin admin] PATH_INFO: `/v3/projects/c2edef81c7ce4f0ba0a52d30a926883c` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:45.870543 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fb9062-ae53-4a50-8082-eba7b97c95ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2edef81c7ce4f0ba0a52d30a926883c)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:45.872989 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fb9062-ae53-4a50-8082-eba7b97c95ec admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:45.963761 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a6fb9062-ae53-4a50-8082-eba7b97c95ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2edef81c7ce4f0ba0a52d30a926883c', 'request_id': 'req-a6fb9062-ae53-4a50-8082-eba7b97c95ec'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:45.971593 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a6fb9062-ae53-4a50-8082-eba7b97c95ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2edef81c7ce4f0ba0a52d30a926883c', 'request_id': 'req-a6fb9062-ae53-4a50-8082-eba7b97c95ec'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:45.972879 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:45.981307 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a6fb9062-ae53-4a50-8082-eba7b97c95ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2edef81c7ce4f0ba0a52d30a926883c', 'request_id': 'req-a6fb9062-ae53-4a50-8082-eba7b97c95ec'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:46.003321 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a6fb9062-ae53-4a50-8082-eba7b97c95ec admin admin] The token cache is being invalidate because project c2edef81c7ce4f0ba0a52d30a926883c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:15:46.003782 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a6fb9062-ae53-4a50-8082-eba7b97c95ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:46.004593 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a6fb9062-ae53-4a50-8082-eba7b97c95ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:46.009297 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1039/2103] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:15:45 2022] DELETE /identity/v3/projects/c2edef81c7ce4f0ba0a52d30a926883c => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:46.010700 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09ee753-577c-43a5-b82c-09acf0b62b1a service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:46.021043 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09ee753-577c-43a5-b82c-09acf0b62b1a service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:46.078727 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:46.107282 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340e1dd2-2f06-44f7-a849-e2bad968f5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:46.108179 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e1dd2-2f06-44f7-a849-e2bad968f5ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:46.108494 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e1dd2-2f06-44f7-a849-e2bad968f5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:46.108928 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e1dd2-2f06-44f7-a849-e2bad968f5ab admin admin] PATH_INFO: `/v3/users/721c5a796c38430fb406a242aa053274` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:46.110405 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340e1dd2-2f06-44f7-a849-e2bad968f5ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=721c5a796c38430fb406a242aa053274)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:46.112341 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340e1dd2-2f06-44f7-a849-e2bad968f5ab admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:46.198278 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-340e1dd2-2f06-44f7-a849-e2bad968f5ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '721c5a796c38430fb406a242aa053274'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:46.218265 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-340e1dd2-2f06-44f7-a849-e2bad968f5ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '721c5a796c38430fb406a242aa053274'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:46.219128 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-340e1dd2-2f06-44f7-a849-e2bad968f5ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '721c5a796c38430fb406a242aa053274'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:46.243509 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9fdf58-4f7b-4bd4-b4f1-6bf94bc7a26c None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:46.243807 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9fdf58-4f7b-4bd4-b4f1-6bf94bc7a26c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:46.244090 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9fdf58-4f7b-4bd4-b4f1-6bf94bc7a26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:46.246908 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-340e1dd2-2f06-44f7-a849-e2bad968f5ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '721c5a796c38430fb406a242aa053274'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:46.248105 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-340e1dd2-2f06-44f7-a849-e2bad968f5ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '721c5a796c38430fb406a242aa053274'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:46.274114 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1065/2104] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:15:46 2022] DELETE /identity/v3/users/721c5a796c38430fb406a242aa053274 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:46.281152 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-6b9fdf58-4f7b-4bd4-b4f1-6bf94bc7a26c None None] MFA Rules not processed for user `34ac2a8ad2e343838f8e3e3285ec5f55`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:46.282672 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6b9fdf58-4f7b-4bd4-b4f1-6bf94bc7a26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:46.350342 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340e1dd2-2f06-44f7-a849-e2bad968f5ab admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:46.353062 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-340e1dd2-2f06-44f7-a849-e2bad968f5ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:46.372471 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1040/2105] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:15:46.520246 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340e1dd2-2f06-44f7-a849-e2bad968f5ab admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:46.531570 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42a2126-0b60-4cce-a1b8-f0a4eeeddce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:46.532507 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a2126-0b60-4cce-a1b8-f0a4eeeddce7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:46.532810 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a2126-0b60-4cce-a1b8-f0a4eeeddce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:46.533093 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a2126-0b60-4cce-a1b8-f0a4eeeddce7 admin admin] PATH_INFO: `/v3/users/92e368db026e4a8f8f4730710471868b` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:46.538457 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42a2126-0b60-4cce-a1b8-f0a4eeeddce7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92e368db026e4a8f8f4730710471868b)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:46.540457 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42a2126-0b60-4cce-a1b8-f0a4eeeddce7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:46.614986 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b42a2126-0b60-4cce-a1b8-f0a4eeeddce7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92e368db026e4a8f8f4730710471868b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:46.626486 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b42a2126-0b60-4cce-a1b8-f0a4eeeddce7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92e368db026e4a8f8f4730710471868b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:46.627430 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b42a2126-0b60-4cce-a1b8-f0a4eeeddce7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92e368db026e4a8f8f4730710471868b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:46.631933 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:46.639016 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b42a2126-0b60-4cce-a1b8-f0a4eeeddce7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92e368db026e4a8f8f4730710471868b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:46.639846 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b42a2126-0b60-4cce-a1b8-f0a4eeeddce7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92e368db026e4a8f8f4730710471868b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:46.663913 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1066/2106] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:15:46 2022] DELETE /identity/v3/users/92e368db026e4a8f8f4730710471868b => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:15:46.980771 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a98c4-10dc-463d-8ba0-af15a215bff3 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:46.981315 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a98c4-10dc-463d-8ba0-af15a215bff3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:46.981315 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a98c4-10dc-463d-8ba0-af15a215bff3 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:46.982830 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1041/2107] 10.222.0.50 () {54 vars in 912 bytes} [Fri May 27 15:15:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:15:46.997420 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e437ae87-b2b8-4328-9a24-8cde3093d9a9 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:46.997799 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e437ae87-b2b8-4328-9a24-8cde3093d9a9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:46.998165 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e437ae87-b2b8-4328-9a24-8cde3093d9a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:47.027248 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e437ae87-b2b8-4328-9a24-8cde3093d9a9 None None] MFA Rules not processed for user `f874edc08438434189f1d14d79291487`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:47.030620 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e437ae87-b2b8-4328-9a24-8cde3093d9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:47.100863 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1067/2108] 10.222.0.50 () {58 vars in 1009 bytes} [Fri May 27 15:15:46 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:47.117181 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929a98c4-10dc-463d-8ba0-af15a215bff3 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:47.120835 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-929a98c4-10dc-463d-8ba0-af15a215bff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:47.132818 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e437ae87-b2b8-4328-9a24-8cde3093d9a9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:47.134320 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e437ae87-b2b8-4328-9a24-8cde3093d9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:47.142421 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929a98c4-10dc-463d-8ba0-af15a215bff3 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:47.153605 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d5bcb8-ffd2-4a76-87b4-136a597dd396 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:47.155512 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5bcb8-ffd2-4a76-87b4-136a597dd396 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:47.156125 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5bcb8-ffd2-4a76-87b4-136a597dd396 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:47.156619 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d5bcb8-ffd2-4a76-87b4-136a597dd396 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:47.174586 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d5bcb8-ffd2-4a76-87b4-136a597dd396 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:47.177119 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d5bcb8-ffd2-4a76-87b4-136a597dd396 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:47.190480 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1042/2109] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:47.201549 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d5bcb8-ffd2-4a76-87b4-136a597dd396 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:47.203716 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c3d5bcb8-ffd2-4a76-87b4-136a597dd396 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:47.305582 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e437ae87-b2b8-4328-9a24-8cde3093d9a9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:47.321253 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93af3a23-99e0-4a3f-a05c-64be70c87b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:47.322177 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af3a23-99e0-4a3f-a05c-64be70c87b75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:47.323000 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af3a23-99e0-4a3f-a05c-64be70c87b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:47.323410 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93af3a23-99e0-4a3f-a05c-64be70c87b75 admin admin] PATH_INFO: `/v3/projects/c89b8275371547799668087fdf49eb56` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:47.325155 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93af3a23-99e0-4a3f-a05c-64be70c87b75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c89b8275371547799668087fdf49eb56)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:47.330296 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93af3a23-99e0-4a3f-a05c-64be70c87b75 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:47.360104 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d5bcb8-ffd2-4a76-87b4-136a597dd396 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:47.366624 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc8da7-55fa-4695-b8ca-7f88ba607da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:47.367647 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc8da7-55fa-4695-b8ca-7f88ba607da0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:47.368043 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc8da7-55fa-4695-b8ca-7f88ba607da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:47.368470 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc8da7-55fa-4695-b8ca-7f88ba607da0 admin admin] PATH_INFO: `/v3/projects/d624b45721514810b49fbbd38c7afb5c` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:47.370997 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dc8da7-55fa-4695-b8ca-7f88ba607da0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d624b45721514810b49fbbd38c7afb5c)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:47.374109 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dc8da7-55fa-4695-b8ca-7f88ba607da0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:47.449836 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-93af3a23-99e0-4a3f-a05c-64be70c87b75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c89b8275371547799668087fdf49eb56', 'request_id': 'req-93af3a23-99e0-4a3f-a05c-64be70c87b75'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:47.464228 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-93af3a23-99e0-4a3f-a05c-64be70c87b75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c89b8275371547799668087fdf49eb56', 'request_id': 'req-93af3a23-99e0-4a3f-a05c-64be70c87b75'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:47.467792 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-34dc8da7-55fa-4695-b8ca-7f88ba607da0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd624b45721514810b49fbbd38c7afb5c', 'request_id': 'req-34dc8da7-55fa-4695-b8ca-7f88ba607da0'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:47.474630 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-93af3a23-99e0-4a3f-a05c-64be70c87b75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c89b8275371547799668087fdf49eb56', 'request_id': 'req-93af3a23-99e0-4a3f-a05c-64be70c87b75'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:47.481783 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-34dc8da7-55fa-4695-b8ca-7f88ba607da0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd624b45721514810b49fbbd38c7afb5c', 'request_id': 'req-34dc8da7-55fa-4695-b8ca-7f88ba607da0'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:47.487982 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-93af3a23-99e0-4a3f-a05c-64be70c87b75 admin admin] The token cache is being invalidate because project c89b8275371547799668087fdf49eb56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:15:47.488368 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-93af3a23-99e0-4a3f-a05c-64be70c87b75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:47.489431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-93af3a23-99e0-4a3f-a05c-64be70c87b75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:47.491499 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1068/2110] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:15:47 2022] DELETE /identity/v3/projects/c89b8275371547799668087fdf49eb56 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:15:47.496113 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93af3a23-99e0-4a3f-a05c-64be70c87b75 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:47.496985 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-34dc8da7-55fa-4695-b8ca-7f88ba607da0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd624b45721514810b49fbbd38c7afb5c', 'request_id': 'req-34dc8da7-55fa-4695-b8ca-7f88ba607da0'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:47.498461 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-93af3a23-99e0-4a3f-a05c-64be70c87b75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:47.513096 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-34dc8da7-55fa-4695-b8ca-7f88ba607da0 admin admin] The token cache is being invalidate because project d624b45721514810b49fbbd38c7afb5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:15:47.513677 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-34dc8da7-55fa-4695-b8ca-7f88ba607da0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:47.514470 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-34dc8da7-55fa-4695-b8ca-7f88ba607da0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:47.516305 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1043/2111] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:15:47 2022] DELETE /identity/v3/projects/d624b45721514810b49fbbd38c7afb5c => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:47.523128 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5019a6-a8ee-451c-8a50-53d1c8e41975 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:47.523448 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5019a6-a8ee-451c-8a50-53d1c8e41975 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:47.523693 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5019a6-a8ee-451c-8a50-53d1c8e41975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:47.537081 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:47.586293 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ce5019a6-a8ee-451c-8a50-53d1c8e41975 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:47.587804 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ce5019a6-a8ee-451c-8a50-53d1c8e41975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:47.597621 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:47.659587 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93af3a23-99e0-4a3f-a05c-64be70c87b75 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:47.730904 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1044/2112] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:47.739058 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce48c4-4a38-440a-8cad-e2646ac78132 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:47.739783 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce48c4-4a38-440a-8cad-e2646ac78132 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:47.740289 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce48c4-4a38-440a-8cad-e2646ac78132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:47.768464 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3385d5be-5807-42f5-9600-7cdda96a22ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:47.769363 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385d5be-5807-42f5-9600-7cdda96a22ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:47.769644 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385d5be-5807-42f5-9600-7cdda96a22ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:47.769922 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3385d5be-5807-42f5-9600-7cdda96a22ac admin admin] PATH_INFO: `/v3/users/33ba679717874332b0364471cfded983` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:47.771196 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3385d5be-5807-42f5-9600-7cdda96a22ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=33ba679717874332b0364471cfded983)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:47.773157 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3385d5be-5807-42f5-9600-7cdda96a22ac admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:47.781258 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-77ce48c4-4a38-440a-8cad-e2646ac78132 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:47.782596 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-77ce48c4-4a38-440a-8cad-e2646ac78132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:47.794734 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1045/2113] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:15:47.813626 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce48c4-4a38-440a-8cad-e2646ac78132 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:47.836661 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce48c4-4a38-440a-8cad-e2646ac78132 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:47.854330 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503cd86-511d-4e8f-901c-5e33acd6dbc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:47.855462 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503cd86-511d-4e8f-901c-5e33acd6dbc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:47.855825 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503cd86-511d-4e8f-901c-5e33acd6dbc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:47.856184 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503cd86-511d-4e8f-901c-5e33acd6dbc7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:47.857841 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1503cd86-511d-4e8f-901c-5e33acd6dbc7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:15:47.862474 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1503cd86-511d-4e8f-901c-5e33acd6dbc7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:47.862474 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1503cd86-511d-4e8f-901c-5e33acd6dbc7 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:47.871378 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3385d5be-5807-42f5-9600-7cdda96a22ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33ba679717874332b0364471cfded983'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:47.898877 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3385d5be-5807-42f5-9600-7cdda96a22ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33ba679717874332b0364471cfded983'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:47.899840 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3385d5be-5807-42f5-9600-7cdda96a22ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33ba679717874332b0364471cfded983'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:47.905555 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1046/2114] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:15:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:15:47.906691 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503cd86-511d-4e8f-901c-5e33acd6dbc7 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:47.912925 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3385d5be-5807-42f5-9600-7cdda96a22ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33ba679717874332b0364471cfded983'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:47.914376 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3385d5be-5807-42f5-9600-7cdda96a22ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33ba679717874332b0364471cfded983'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:47.919602 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503cd86-511d-4e8f-901c-5e33acd6dbc7 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:47.924848 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3733c12-5f53-4843-a83b-a0f2f2af0886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:47.925789 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3733c12-5f53-4843-a83b-a0f2f2af0886 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:47.926400 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3733c12-5f53-4843-a83b-a0f2f2af0886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:47.926862 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3733c12-5f53-4843-a83b-a0f2f2af0886 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:47.927787 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3733c12-5f53-4843-a83b-a0f2f2af0886 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:47.930098 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3733c12-5f53-4843-a83b-a0f2f2af0886 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:47.942909 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1069/2115] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:15:47 2022] DELETE /identity/v3/users/33ba679717874332b0364471cfded983 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:47.970095 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1047/2116] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:15:47 2022] POST /identity/v3/projects => generated 393 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:47.974091 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3385d5be-5807-42f5-9600-7cdda96a22ac admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:47.975922 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3385d5be-5807-42f5-9600-7cdda96a22ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:48.069399 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:48.119004 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3385d5be-5807-42f5-9600-7cdda96a22ac admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:48.211908 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cd0a93-6b10-40c2-a4f4-26813354c324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:48.213070 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd0a93-6b10-40c2-a4f4-26813354c324 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:48.213556 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd0a93-6b10-40c2-a4f4-26813354c324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:48.214011 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cd0a93-6b10-40c2-a4f4-26813354c324 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:48.215092 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cd0a93-6b10-40c2-a4f4-26813354c324 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:48.217367 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cd0a93-6b10-40c2-a4f4-26813354c324 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:48.246895 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1070/2117] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:15:47 2022] POST /identity/v3/users => generated 297 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:48.251082 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3733c12-5f53-4843-a83b-a0f2f2af0886 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:48.264442 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3733c12-5f53-4843-a83b-a0f2f2af0886 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:48.283939 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59994a4-8559-4bf5-8023-c68597255cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:48.285326 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59994a4-8559-4bf5-8023-c68597255cf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:48.285820 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59994a4-8559-4bf5-8023-c68597255cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:48.286321 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59994a4-8559-4bf5-8023-c68597255cf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:48.288191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59994a4-8559-4bf5-8023-c68597255cf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:48.290043 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59994a4-8559-4bf5-8023-c68597255cf5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:48.305129 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1048/2118] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:48 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:48.310158 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cd0a93-6b10-40c2-a4f4-26813354c324 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:48.325469 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cd0a93-6b10-40c2-a4f4-26813354c324 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:48.335988 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b33ae93-5a86-4866-b4d4-48f7b1615939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:48.337129 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b33ae93-5a86-4866-b4d4-48f7b1615939 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:48.337574 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b33ae93-5a86-4866-b4d4-48f7b1615939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:48.338043 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b33ae93-5a86-4866-b4d4-48f7b1615939 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:48.338988 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b33ae93-5a86-4866-b4d4-48f7b1615939 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:48.340900 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b33ae93-5a86-4866-b4d4-48f7b1615939 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:48.356102 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1071/2119] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:48 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:48.361024 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59994a4-8559-4bf5-8023-c68597255cf5 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:48.376746 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59994a4-8559-4bf5-8023-c68597255cf5 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:48.393707 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eaa39c-92f2-4c2d-a0f6-cbcbd68d2414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:48.394640 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eaa39c-92f2-4c2d-a0f6-cbcbd68d2414 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:48.394953 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eaa39c-92f2-4c2d-a0f6-cbcbd68d2414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:48.395238 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eaa39c-92f2-4c2d-a0f6-cbcbd68d2414 admin admin] PATH_INFO: `/v3/projects/c208c3d86c504ea086bf763e241cbc88/users/9c9149cb2b3c423796164b487a305bbd/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:48.444881 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eaa39c-92f2-4c2d-a0f6-cbcbd68d2414 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c208c3d86c504ea086bf763e241cbc88, user_id=9c9149cb2b3c423796164b487a305bbd, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:48.449067 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eaa39c-92f2-4c2d-a0f6-cbcbd68d2414 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:48.475887 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1049/2120] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:48 2022] PUT /identity/v3/projects/c208c3d86c504ea086bf763e241cbc88/users/9c9149cb2b3c423796164b487a305bbd/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:48.481675 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d212290-9ab1-4480-a264-1339c6282d58 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:48.482203 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d212290-9ab1-4480-a264-1339c6282d58 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:48.482702 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d212290-9ab1-4480-a264-1339c6282d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:48.518089 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3d212290-9ab1-4480-a264-1339c6282d58 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:48.519513 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3d212290-9ab1-4480-a264-1339c6282d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:48.603255 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eaa39c-92f2-4c2d-a0f6-cbcbd68d2414 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:48.610186 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-35eaa39c-92f2-4c2d-a0f6-cbcbd68d2414 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:48.672878 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1072/2121] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:48.689965 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d212290-9ab1-4480-a264-1339c6282d58 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:48.693519 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3d212290-9ab1-4480-a264-1339c6282d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:48.732021 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eaa39c-92f2-4c2d-a0f6-cbcbd68d2414 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:48.737196 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5332f644-7f85-4da3-8860-9169b7065f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:48.738121 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332f644-7f85-4da3-8860-9169b7065f94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:48.738414 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332f644-7f85-4da3-8860-9169b7065f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:48.738686 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332f644-7f85-4da3-8860-9169b7065f94 admin admin] PATH_INFO: `/v3/projects/ec9ecb18f3484a20ae4eb44472a338dd` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:48.740035 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5332f644-7f85-4da3-8860-9169b7065f94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec9ecb18f3484a20ae4eb44472a338dd)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:48.742347 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5332f644-7f85-4da3-8860-9169b7065f94 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:48.788673 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d212290-9ab1-4480-a264-1339c6282d58 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:48.793866 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8740d7-9af1-4ab4-af3c-682979349c43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:48.794536 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8740d7-9af1-4ab4-af3c-682979349c43 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:48.794817 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8740d7-9af1-4ab4-af3c-682979349c43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:48.795322 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8740d7-9af1-4ab4-af3c-682979349c43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:48.806382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:48.812684 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8740d7-9af1-4ab4-af3c-682979349c43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:48.815135 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8740d7-9af1-4ab4-af3c-682979349c43 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:48.827278 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5332f644-7f85-4da3-8860-9169b7065f94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec9ecb18f3484a20ae4eb44472a338dd', 'request_id': 'req-5332f644-7f85-4da3-8860-9169b7065f94'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:48.827731 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1073/2122] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:48.838976 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5332f644-7f85-4da3-8860-9169b7065f94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec9ecb18f3484a20ae4eb44472a338dd', 'request_id': 'req-5332f644-7f85-4da3-8860-9169b7065f94'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:48.848379 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5332f644-7f85-4da3-8860-9169b7065f94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec9ecb18f3484a20ae4eb44472a338dd', 'request_id': 'req-5332f644-7f85-4da3-8860-9169b7065f94'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:48.867932 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5332f644-7f85-4da3-8860-9169b7065f94 admin admin] The token cache is being invalidate because project ec9ecb18f3484a20ae4eb44472a338dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:15:48.868471 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5332f644-7f85-4da3-8860-9169b7065f94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:48.870084 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5332f644-7f85-4da3-8860-9169b7065f94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:48.872363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1050/2123] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:15:48 2022] DELETE /identity/v3/projects/ec9ecb18f3484a20ae4eb44472a338dd => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:15:48.877593 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753bb31-6f4c-4631-b036-e22101bc4e72 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:48.877885 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753bb31-6f4c-4631-b036-e22101bc4e72 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:48.878639 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753bb31-6f4c-4631-b036-e22101bc4e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:48.908745 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9753bb31-6f4c-4631-b036-e22101bc4e72 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:48.910437 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9753bb31-6f4c-4631-b036-e22101bc4e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:48.921887 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:49.116319 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7411ab-543b-4d4d-9071-be34294567c5 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:49.116507 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1074/2124] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:49.116947 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7411ab-543b-4d4d-9071-be34294567c5 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:49.117133 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7411ab-543b-4d4d-9071-be34294567c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:49.172975 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-9d7411ab-543b-4d4d-9071-be34294567c5 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:49.173538 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9d7411ab-543b-4d4d-9071-be34294567c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:49.191413 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1051/2125] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:15:49.195903 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9753bb31-6f4c-4631-b036-e22101bc4e72 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:49.211252 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9753bb31-6f4c-4631-b036-e22101bc4e72 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:49.222875 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f40ea7f-564b-4636-a2f5-fa12a6d5c9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:49.223590 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40ea7f-564b-4636-a2f5-fa12a6d5c9b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:49.223879 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40ea7f-564b-4636-a2f5-fa12a6d5c9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:49.224165 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f40ea7f-564b-4636-a2f5-fa12a6d5c9b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:49.224908 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f40ea7f-564b-4636-a2f5-fa12a6d5c9b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:15:49.225233 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f40ea7f-564b-4636-a2f5-fa12a6d5c9b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:49.226828 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f40ea7f-564b-4636-a2f5-fa12a6d5c9b3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:49.258919 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1075/2126] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:15:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:15:49.264250 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7411ab-543b-4d4d-9071-be34294567c5 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:49.284880 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7411ab-543b-4d4d-9071-be34294567c5 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:49.299912 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa8421-ed23-4937-a75d-3a5127eb997f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:49.321974 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa8421-ed23-4937-a75d-3a5127eb997f admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:49.325112 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa8421-ed23-4937-a75d-3a5127eb997f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:49.326710 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aa8421-ed23-4937-a75d-3a5127eb997f admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:49.328271 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aa8421-ed23-4937-a75d-3a5127eb997f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:49.331896 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aa8421-ed23-4937-a75d-3a5127eb997f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:49.374111 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1052/2127] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:15:49 2022] POST /identity/v3/projects => generated 391 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:49.378549 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f40ea7f-564b-4636-a2f5-fa12a6d5c9b3 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:49.397366 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f40ea7f-564b-4636-a2f5-fa12a6d5c9b3 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:49.556806 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd79670f-62b1-4e1e-b467-7416c3e31139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:49.558667 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79670f-62b1-4e1e-b467-7416c3e31139 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:49.559142 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79670f-62b1-4e1e-b467-7416c3e31139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:49.559595 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd79670f-62b1-4e1e-b467-7416c3e31139 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:49.560495 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd79670f-62b1-4e1e-b467-7416c3e31139 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:49.563253 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd79670f-62b1-4e1e-b467-7416c3e31139 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:49.593963 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1076/2128] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:15:49 2022] POST /identity/v3/users => generated 296 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:15:49.595627 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa8421-ed23-4937-a75d-3a5127eb997f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:49.614111 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aa8421-ed23-4937-a75d-3a5127eb997f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:49.631057 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf604b0-6981-46b4-8491-6a3521feb536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:49.632941 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf604b0-6981-46b4-8491-6a3521feb536 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:49.632941 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf604b0-6981-46b4-8491-6a3521feb536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:49.632941 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf604b0-6981-46b4-8491-6a3521feb536 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:49.633233 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf604b0-6981-46b4-8491-6a3521feb536 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:49.647071 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf604b0-6981-46b4-8491-6a3521feb536 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:49.679453 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1053/2129] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:49 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:49.681004 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd79670f-62b1-4e1e-b467-7416c3e31139 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:49.699184 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd79670f-62b1-4e1e-b467-7416c3e31139 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:49.718942 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cd715c-485d-42de-99df-5334937094ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:49.719812 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd715c-485d-42de-99df-5334937094ce admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:49.720192 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd715c-485d-42de-99df-5334937094ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:49.720563 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd715c-485d-42de-99df-5334937094ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:49.721727 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cd715c-485d-42de-99df-5334937094ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:49.724034 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cd715c-485d-42de-99df-5334937094ce admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:49.763236 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1077/2130] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:49 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:15:49.767607 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf604b0-6981-46b4-8491-6a3521feb536 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:49.782321 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf604b0-6981-46b4-8491-6a3521feb536 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:49.788910 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e958fa3-622e-42d8-a9e3-ad449a3e50ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:49.790131 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e958fa3-622e-42d8-a9e3-ad449a3e50ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:49.790873 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e958fa3-622e-42d8-a9e3-ad449a3e50ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:49.791380 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e958fa3-622e-42d8-a9e3-ad449a3e50ce admin admin] PATH_INFO: `/v3/projects/5de7b2affd3f432081e89391e3adce99/users/fd592734e7124bb8b1fdc7b351ae018b/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:49.812606 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e958fa3-622e-42d8-a9e3-ad449a3e50ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=5de7b2affd3f432081e89391e3adce99, user_id=fd592734e7124bb8b1fdc7b351ae018b, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:49.816720 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e958fa3-622e-42d8-a9e3-ad449a3e50ce admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:49.861593 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1054/2131] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:49 2022] PUT /identity/v3/projects/5de7b2affd3f432081e89391e3adce99/users/fd592734e7124bb8b1fdc7b351ae018b/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:49.876525 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed02da3-4c0a-41dc-a168-bda3c306a3b9 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:49.877413 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed02da3-4c0a-41dc-a168-bda3c306a3b9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:49.878290 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed02da3-4c0a-41dc-a168-bda3c306a3b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:49.969740 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8ed02da3-4c0a-41dc-a168-bda3c306a3b9 None None] MFA Rules not processed for user `fd592734e7124bb8b1fdc7b351ae018b`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:49.975476 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8ed02da3-4c0a-41dc-a168-bda3c306a3b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:50.153790 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1078/2132] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:50.171787 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e958fa3-622e-42d8-a9e3-ad449a3e50ce admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:50.173166 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8e958fa3-622e-42d8-a9e3-ad449a3e50ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:50.388819 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e958fa3-622e-42d8-a9e3-ad449a3e50ce admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:50.400731 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff354d-47d6-42bd-bfad-e75a2443ae4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:50.402063 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff354d-47d6-42bd-bfad-e75a2443ae4b service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:50.402430 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff354d-47d6-42bd-bfad-e75a2443ae4b service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:50.402814 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff354d-47d6-42bd-bfad-e75a2443ae4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:50.422250 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ff354d-47d6-42bd-bfad-e75a2443ae4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:50.426788 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ff354d-47d6-42bd-bfad-e75a2443ae4b service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:50.455184 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1055/2133] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:50 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:53.586130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed02da3-4c0a-41dc-a168-bda3c306a3b9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:53.589351 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8ed02da3-4c0a-41dc-a168-bda3c306a3b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:53.609951 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:53.712152 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed02da3-4c0a-41dc-a168-bda3c306a3b9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:53.717382 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c313fc7d-c8bf-4e73-8311-978a557d2332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:53.718524 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c313fc7d-c8bf-4e73-8311-978a557d2332 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:53.718800 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c313fc7d-c8bf-4e73-8311-978a557d2332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:53.719207 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c313fc7d-c8bf-4e73-8311-978a557d2332 admin admin] PATH_INFO: `/v3/users/a9a2b077f6704778bf4090b555a02c60` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:53.723909 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c313fc7d-c8bf-4e73-8311-978a557d2332 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9a2b077f6704778bf4090b555a02c60)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:53.726009 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c313fc7d-c8bf-4e73-8311-978a557d2332 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:53.789937 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c313fc7d-c8bf-4e73-8311-978a557d2332 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9a2b077f6704778bf4090b555a02c60'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:53.798828 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c313fc7d-c8bf-4e73-8311-978a557d2332 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9a2b077f6704778bf4090b555a02c60'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:53.800472 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c313fc7d-c8bf-4e73-8311-978a557d2332 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9a2b077f6704778bf4090b555a02c60'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:53.810671 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c313fc7d-c8bf-4e73-8311-978a557d2332 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9a2b077f6704778bf4090b555a02c60'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:53.811342 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c313fc7d-c8bf-4e73-8311-978a557d2332 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9a2b077f6704778bf4090b555a02c60'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:53.835630 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1079/2134] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:15:53 2022] DELETE /identity/v3/users/a9a2b077f6704778bf4090b555a02c60 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:53.840452 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd78b21-ded0-4dde-bfb2-5fa798fac8fe None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:53.841001 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd78b21-ded0-4dde-bfb2-5fa798fac8fe None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:53.841101 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd78b21-ded0-4dde-bfb2-5fa798fac8fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:53.881827 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-cbd78b21-ded0-4dde-bfb2-5fa798fac8fe None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:53.884406 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cbd78b21-ded0-4dde-bfb2-5fa798fac8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:54.010209 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1056/2135] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:54.020785 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c313fc7d-c8bf-4e73-8311-978a557d2332 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:54.022382 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c313fc7d-c8bf-4e73-8311-978a557d2332 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:54.111078 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c313fc7d-c8bf-4e73-8311-978a557d2332 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:54.115191 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2652cbb-b552-49a9-af7c-5ff8cd76ff47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:54.115980 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2652cbb-b552-49a9-af7c-5ff8cd76ff47 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:54.116417 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2652cbb-b552-49a9-af7c-5ff8cd76ff47 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:54.116797 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2652cbb-b552-49a9-af7c-5ff8cd76ff47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:54.132304 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2652cbb-b552-49a9-af7c-5ff8cd76ff47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:54.134132 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2652cbb-b552-49a9-af7c-5ff8cd76ff47 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:54.171695 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1080/2136] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:54.457946 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221b562-c99c-449b-994d-0d9bad7f65ac None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:54.458239 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221b562-c99c-449b-994d-0d9bad7f65ac None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:54.458686 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3221b562-c99c-449b-994d-0d9bad7f65ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:54.467280 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:54.502529 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3221b562-c99c-449b-994d-0d9bad7f65ac None None] MFA Rules not processed for user `50ca9d699b004df4becfe3241802fb17`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:54.504278 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3221b562-c99c-449b-994d-0d9bad7f65ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:54.673623 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1057/2137] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:54.677847 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2652cbb-b552-49a9-af7c-5ff8cd76ff47 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:54.679742 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d2652cbb-b552-49a9-af7c-5ff8cd76ff47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:54.704257 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2652cbb-b552-49a9-af7c-5ff8cd76ff47 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:54.715008 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c654c6-6dd8-4604-a583-219c7d6a80f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:54.717322 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c654c6-6dd8-4604-a583-219c7d6a80f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:54.717322 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c654c6-6dd8-4604-a583-219c7d6a80f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:54.719094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c654c6-6dd8-4604-a583-219c7d6a80f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:54.720083 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c654c6-6dd8-4604-a583-219c7d6a80f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:54.722511 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c654c6-6dd8-4604-a583-219c7d6a80f9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:54.769851 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3221b562-c99c-449b-994d-0d9bad7f65ac None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:54.770421 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1081/2138] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:15:54 2022] POST /identity/v3/projects => generated 385 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:54.801602 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3221b562-c99c-449b-994d-0d9bad7f65ac None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:54.941816 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d27c697-4b96-48bf-9ee5-82760328d931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:54.943987 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27c697-4b96-48bf-9ee5-82760328d931 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:54.948160 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27c697-4b96-48bf-9ee5-82760328d931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:54.948713 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27c697-4b96-48bf-9ee5-82760328d931 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:54.950101 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d27c697-4b96-48bf-9ee5-82760328d931 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:54.957036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d27c697-4b96-48bf-9ee5-82760328d931 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:55.007684 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1058/2139] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:15:54 2022] POST /identity/v3/users => generated 299 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:15:55.012037 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c654c6-6dd8-4604-a583-219c7d6a80f9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:55.025553 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c654c6-6dd8-4604-a583-219c7d6a80f9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:55.040946 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d81023-9707-485a-aaa4-5022099a9a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:55.043969 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d81023-9707-485a-aaa4-5022099a9a69 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:55.048062 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d81023-9707-485a-aaa4-5022099a9a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:55.049030 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d81023-9707-485a-aaa4-5022099a9a69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:55.066774 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d81023-9707-485a-aaa4-5022099a9a69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:55.068785 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d81023-9707-485a-aaa4-5022099a9a69 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:55.076061 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:55.101770 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1082/2140] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:15:55.105800 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d27c697-4b96-48bf-9ee5-82760328d931 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:55.122585 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d27c697-4b96-48bf-9ee5-82760328d931 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:55.128164 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d81023-9707-485a-aaa4-5022099a9a69 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:55.131277 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-03d81023-9707-485a-aaa4-5022099a9a69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:55.134066 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b381d-1cf4-467c-a6ff-19ac9540fc58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:55.135622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b381d-1cf4-467c-a6ff-19ac9540fc58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:55.139256 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b381d-1cf4-467c-a6ff-19ac9540fc58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:55.139809 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b381d-1cf4-467c-a6ff-19ac9540fc58 admin admin] PATH_INFO: `/v3/projects/a4313b863b7f453587d5708db79e2143/users/18ff5bfbc9f8453b981fb228ca2c1b6c/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:55.167295 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d81023-9707-485a-aaa4-5022099a9a69 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:55.177913 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6c8c3-345b-4764-8df4-30d8ce6496c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:55.179815 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6c8c3-345b-4764-8df4-30d8ce6496c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:55.180595 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6c8c3-345b-4764-8df4-30d8ce6496c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:55.181305 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d6c8c3-345b-4764-8df4-30d8ce6496c1 admin admin] PATH_INFO: `/v3/projects/703969aecb654b3f9fff32e41d8f8f09` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:55.183704 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475b381d-1cf4-467c-a6ff-19ac9540fc58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4313b863b7f453587d5708db79e2143, user_id=18ff5bfbc9f8453b981fb228ca2c1b6c, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:55.184592 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d6c8c3-345b-4764-8df4-30d8ce6496c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=703969aecb654b3f9fff32e41d8f8f09)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:55.191040 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d6c8c3-345b-4764-8df4-30d8ce6496c1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:55.192559 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475b381d-1cf4-467c-a6ff-19ac9540fc58 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:55.219275 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1059/2141] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:55 2022] PUT /identity/v3/projects/a4313b863b7f453587d5708db79e2143/users/18ff5bfbc9f8453b981fb228ca2c1b6c/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:55.224291 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b381d-1cf4-467c-a6ff-19ac9540fc58 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:55.324982 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-25d6c8c3-345b-4764-8df4-30d8ce6496c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '703969aecb654b3f9fff32e41d8f8f09', 'request_id': 'req-25d6c8c3-345b-4764-8df4-30d8ce6496c1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:55.342825 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-25d6c8c3-345b-4764-8df4-30d8ce6496c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '703969aecb654b3f9fff32e41d8f8f09', 'request_id': 'req-25d6c8c3-345b-4764-8df4-30d8ce6496c1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:55.353785 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-25d6c8c3-345b-4764-8df4-30d8ce6496c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '703969aecb654b3f9fff32e41d8f8f09', 'request_id': 'req-25d6c8c3-345b-4764-8df4-30d8ce6496c1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:55.381846 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-25d6c8c3-345b-4764-8df4-30d8ce6496c1 admin admin] The token cache is being invalidate because project 703969aecb654b3f9fff32e41d8f8f09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:15:55.382183 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-25d6c8c3-345b-4764-8df4-30d8ce6496c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:55.385406 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-25d6c8c3-345b-4764-8df4-30d8ce6496c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:55.390784 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:55.390970 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1083/2142] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:15:55 2022] DELETE /identity/v3/projects/703969aecb654b3f9fff32e41d8f8f09 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:55.401742 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b381d-1cf4-467c-a6ff-19ac9540fc58 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:55.404502 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6c8c3-345b-4764-8df4-30d8ce6496c1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:55.408917 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-25d6c8c3-345b-4764-8df4-30d8ce6496c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:55.523797 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d6c8c3-345b-4764-8df4-30d8ce6496c1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:55.530390 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4ae1d5-cc96-4dd9-ad18-13ea5416d527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:55.531241 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ae1d5-cc96-4dd9-ad18-13ea5416d527 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:55.531535 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ae1d5-cc96-4dd9-ad18-13ea5416d527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:55.531885 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4ae1d5-cc96-4dd9-ad18-13ea5416d527 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:55.532674 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4ae1d5-cc96-4dd9-ad18-13ea5416d527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:55.533823 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e186e5c-e269-4828-9647-7da76ec908bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:55.534559 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4ae1d5-cc96-4dd9-ad18-13ea5416d527 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:55.544537 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e186e5c-e269-4828-9647-7da76ec908bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:55.545818 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e186e5c-e269-4828-9647-7da76ec908bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:55.546331 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e186e5c-e269-4828-9647-7da76ec908bf admin admin] PATH_INFO: `/v3/users/39abb37fd92a4985a1c694d9e83ae8c8` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:55.548326 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e186e5c-e269-4828-9647-7da76ec908bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=39abb37fd92a4985a1c694d9e83ae8c8)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:55.550772 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e186e5c-e269-4828-9647-7da76ec908bf admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:55.560507 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1060/2143] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:55.564643 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4ae1d5-cc96-4dd9-ad18-13ea5416d527 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:55.570725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7d4ae1d5-cc96-4dd9-ad18-13ea5416d527 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:55.595954 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4ae1d5-cc96-4dd9-ad18-13ea5416d527 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:55.612361 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba7a4e4-7d0b-4b07-b98b-c4e443784923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:55.633462 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba7a4e4-7d0b-4b07-b98b-c4e443784923 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:55.633798 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba7a4e4-7d0b-4b07-b98b-c4e443784923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:55.636443 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba7a4e4-7d0b-4b07-b98b-c4e443784923 admin admin] PATH_INFO: `/v3/projects/a4313b863b7f453587d5708db79e2143/users/18ff5bfbc9f8453b981fb228ca2c1b6c/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:55.641031 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba7a4e4-7d0b-4b07-b98b-c4e443784923 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4313b863b7f453587d5708db79e2143, user_id=18ff5bfbc9f8453b981fb228ca2c1b6c, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:55.645697 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba7a4e4-7d0b-4b07-b98b-c4e443784923 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:55.683794 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7e186e5c-e269-4828-9647-7da76ec908bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39abb37fd92a4985a1c694d9e83ae8c8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:55.693192 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1061/2144] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:55 2022] PUT /identity/v3/projects/a4313b863b7f453587d5708db79e2143/users/18ff5bfbc9f8453b981fb228ca2c1b6c/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:15:55.706241 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7e186e5c-e269-4828-9647-7da76ec908bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39abb37fd92a4985a1c694d9e83ae8c8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:55.707283 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7e186e5c-e269-4828-9647-7da76ec908bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39abb37fd92a4985a1c694d9e83ae8c8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:55.719893 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7e186e5c-e269-4828-9647-7da76ec908bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39abb37fd92a4985a1c694d9e83ae8c8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:55.722116 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7e186e5c-e269-4828-9647-7da76ec908bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39abb37fd92a4985a1c694d9e83ae8c8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:55.750697 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:55.766959 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1084/2145] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:15:55 2022] DELETE /identity/v3/users/39abb37fd92a4985a1c694d9e83ae8c8 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:55.923144 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dbfb52-f751-450e-a00e-3cb37aaaac9b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:55.923783 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dbfb52-f751-450e-a00e-3cb37aaaac9b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:55.924250 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dbfb52-f751-450e-a00e-3cb37aaaac9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:56.014514 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e2dbfb52-f751-450e-a00e-3cb37aaaac9b None None] MFA Rules not processed for user `9c9149cb2b3c423796164b487a305bbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:56.029920 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e2dbfb52-f751-450e-a00e-3cb37aaaac9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:56.184251 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1062/2146] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:55 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:56.186047 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395bb594-ab44-4d11-a826-37e8016f80de None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:56.186343 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395bb594-ab44-4d11-a826-37e8016f80de None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:56.186626 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395bb594-ab44-4d11-a826-37e8016f80de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:56.215806 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-395bb594-ab44-4d11-a826-37e8016f80de None None] MFA Rules not processed for user `9c9149cb2b3c423796164b487a305bbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:56.217163 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-395bb594-ab44-4d11-a826-37e8016f80de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:56.226183 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1085/2147] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:56.239440 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dbfb52-f751-450e-a00e-3cb37aaaac9b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:56.241228 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e2dbfb52-f751-450e-a00e-3cb37aaaac9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:56.394061 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dbfb52-f751-450e-a00e-3cb37aaaac9b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:56.403002 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a23120-d4bb-40ca-a074-04b2f921475e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '451a9232f95e494683d81b6078a318d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:56.404492 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a23120-d4bb-40ca-a074-04b2f921475e service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:56.405068 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a23120-d4bb-40ca-a074-04b2f921475e service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:56.405535 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a23120-d4bb-40ca-a074-04b2f921475e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:56.424226 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a23120-d4bb-40ca-a074-04b2f921475e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:56.426384 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a23120-d4bb-40ca-a074-04b2f921475e service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:56.452859 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1063/2148] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:15:56.737279 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395bb594-ab44-4d11-a826-37e8016f80de None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:56.740513 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-395bb594-ab44-4d11-a826-37e8016f80de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:56.887432 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395bb594-ab44-4d11-a826-37e8016f80de None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:56.897125 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed29e6c-cc50-47e8-b0a5-b55d27f50cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:56.898283 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed29e6c-cc50-47e8-b0a5-b55d27f50cd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:56.898697 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed29e6c-cc50-47e8-b0a5-b55d27f50cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:56.899126 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed29e6c-cc50-47e8-b0a5-b55d27f50cd3 admin admin] PATH_INFO: `/v3/projects/e43c1071d395445db48c7a5c4adb24b0` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:56.902105 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed29e6c-cc50-47e8-b0a5-b55d27f50cd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e43c1071d395445db48c7a5c4adb24b0)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:56.904246 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed29e6c-cc50-47e8-b0a5-b55d27f50cd3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:57.034405 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9ed29e6c-cc50-47e8-b0a5-b55d27f50cd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e43c1071d395445db48c7a5c4adb24b0', 'request_id': 'req-9ed29e6c-cc50-47e8-b0a5-b55d27f50cd3'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:57.046612 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9ed29e6c-cc50-47e8-b0a5-b55d27f50cd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e43c1071d395445db48c7a5c4adb24b0', 'request_id': 'req-9ed29e6c-cc50-47e8-b0a5-b55d27f50cd3'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:57.060871 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9ed29e6c-cc50-47e8-b0a5-b55d27f50cd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e43c1071d395445db48c7a5c4adb24b0', 'request_id': 'req-9ed29e6c-cc50-47e8-b0a5-b55d27f50cd3'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:57.081128 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9ed29e6c-cc50-47e8-b0a5-b55d27f50cd3 admin admin] The token cache is being invalidate because project e43c1071d395445db48c7a5c4adb24b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:15:57.081632 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9ed29e6c-cc50-47e8-b0a5-b55d27f50cd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:57.083073 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9ed29e6c-cc50-47e8-b0a5-b55d27f50cd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:15:57.086495 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1086/2149] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:15:56 2022] DELETE /identity/v3/projects/e43c1071d395445db48c7a5c4adb24b0 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:57.092482 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7359f-aec7-4dd9-90b9-1b5639c9ad3a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:57.093730 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7359f-aec7-4dd9-90b9-1b5639c9ad3a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:57.094086 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7359f-aec7-4dd9-90b9-1b5639c9ad3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:57.108281 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:57.153374 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-d9a7359f-aec7-4dd9-90b9-1b5639c9ad3a None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:57.155292 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d9a7359f-aec7-4dd9-90b9-1b5639c9ad3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:57.267585 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1064/2150] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:15:57.269281 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b10f2-5033-47e3-9387-ef87d85cb9f7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:57.269572 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b10f2-5033-47e3-9387-ef87d85cb9f7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:57.269853 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500b10f2-5033-47e3-9387-ef87d85cb9f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:57.297522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-500b10f2-5033-47e3-9387-ef87d85cb9f7 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:57.298745 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-500b10f2-5033-47e3-9387-ef87d85cb9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:57.307941 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1087/2151] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:57.312717 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a7359f-aec7-4dd9-90b9-1b5639c9ad3a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:57.325264 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a7359f-aec7-4dd9-90b9-1b5639c9ad3a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:57.331945 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1330cb1c-cde6-48df-b766-4f15e5c7ca15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:57.332685 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1330cb1c-cde6-48df-b766-4f15e5c7ca15 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:57.333044 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1330cb1c-cde6-48df-b766-4f15e5c7ca15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:57.333479 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1330cb1c-cde6-48df-b766-4f15e5c7ca15 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:57.334680 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1330cb1c-cde6-48df-b766-4f15e5c7ca15 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:15:57.335070 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1330cb1c-cde6-48df-b766-4f15e5c7ca15 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:57.337097 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1330cb1c-cde6-48df-b766-4f15e5c7ca15 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:57.356411 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1065/2152] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:15:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:15:57.358937 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b10f2-5033-47e3-9387-ef87d85cb9f7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:57.374676 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500b10f2-5033-47e3-9387-ef87d85cb9f7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:57.381307 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd8b6b5-cdd4-4014-9c70-959376a0a8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:57.382027 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8b6b5-cdd4-4014-9c70-959376a0a8f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:57.382311 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8b6b5-cdd4-4014-9c70-959376a0a8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:57.382587 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd8b6b5-cdd4-4014-9c70-959376a0a8f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:57.383372 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd8b6b5-cdd4-4014-9c70-959376a0a8f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:57.389181 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd8b6b5-cdd4-4014-9c70-959376a0a8f0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:57.430103 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1088/2153] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:15:57 2022] POST /identity/v3/projects => generated 373 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:57.435859 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1330cb1c-cde6-48df-b766-4f15e5c7ca15 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:57.450238 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1330cb1c-cde6-48df-b766-4f15e5c7ca15 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:57.566281 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62becc2a-1380-4f65-a0cf-28ea574ef766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:57.567767 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62becc2a-1380-4f65-a0cf-28ea574ef766 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:57.568064 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62becc2a-1380-4f65-a0cf-28ea574ef766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:57.568351 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62becc2a-1380-4f65-a0cf-28ea574ef766 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:57.569215 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62becc2a-1380-4f65-a0cf-28ea574ef766 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:57.571637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62becc2a-1380-4f65-a0cf-28ea574ef766 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:57.616756 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd8b6b5-cdd4-4014-9c70-959376a0a8f0 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:57.623107 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1066/2154] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:15:57 2022] POST /identity/v3/users => generated 287 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:15:57.651237 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd8b6b5-cdd4-4014-9c70-959376a0a8f0 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:57.663020 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41490cfc-6f20-44a2-8ef7-ab54449ffce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:57.664101 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41490cfc-6f20-44a2-8ef7-ab54449ffce6 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:57.667019 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41490cfc-6f20-44a2-8ef7-ab54449ffce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:57.668477 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41490cfc-6f20-44a2-8ef7-ab54449ffce6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:57.669623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41490cfc-6f20-44a2-8ef7-ab54449ffce6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:57.673012 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41490cfc-6f20-44a2-8ef7-ab54449ffce6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:57.708986 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1089/2155] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:57 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:15:57.709710 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62becc2a-1380-4f65-a0cf-28ea574ef766 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:57.730380 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62becc2a-1380-4f65-a0cf-28ea574ef766 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:57.750794 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e05205-4419-4371-aa3d-3acc79fedf56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:57.750794 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e05205-4419-4371-aa3d-3acc79fedf56 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:57.750794 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e05205-4419-4371-aa3d-3acc79fedf56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:57.750794 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e05205-4419-4371-aa3d-3acc79fedf56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:57.751338 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e05205-4419-4371-aa3d-3acc79fedf56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:57.760954 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e05205-4419-4371-aa3d-3acc79fedf56 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:57.792131 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1067/2156] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:57 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:15:57.796024 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41490cfc-6f20-44a2-8ef7-ab54449ffce6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:57.827561 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41490cfc-6f20-44a2-8ef7-ab54449ffce6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:57.832584 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ccf540-750b-47a4-96de-14e4805c9980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:57.833497 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ccf540-750b-47a4-96de-14e4805c9980 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:57.833784 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ccf540-750b-47a4-96de-14e4805c9980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:57.834061 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ccf540-750b-47a4-96de-14e4805c9980 admin admin] PATH_INFO: `/v3/projects/f1294c0637814f03880123420bf65b6f/users/c14700617be147a2b6a61df5e8c842f0/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:57.867325 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ccf540-750b-47a4-96de-14e4805c9980 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1294c0637814f03880123420bf65b6f, user_id=c14700617be147a2b6a61df5e8c842f0, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:57.871311 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ccf540-750b-47a4-96de-14e4805c9980 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:57.912955 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1090/2157] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:57 2022] PUT /identity/v3/projects/f1294c0637814f03880123420bf65b6f/users/c14700617be147a2b6a61df5e8c842f0/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:57.924941 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd84876-1482-4085-be25-82a463c22649 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:57.925335 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd84876-1482-4085-be25-82a463c22649 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:57.925507 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd84876-1482-4085-be25-82a463c22649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:57.996030 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5dd84876-1482-4085-be25-82a463c22649 None None] MFA Rules not processed for user `c14700617be147a2b6a61df5e8c842f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:58.014859 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5dd84876-1482-4085-be25-82a463c22649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:58.206446 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:58.209661 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1068/2158] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:58.215653 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ccf540-750b-47a4-96de-14e4805c9980 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:58.278713 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:58.380155 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ccf540-750b-47a4-96de-14e4805c9980 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:58.394093 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b920ea88-8f25-43b1-81ef-aa362767016f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:58.394970 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920ea88-8f25-43b1-81ef-aa362767016f admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:58.396310 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920ea88-8f25-43b1-81ef-aa362767016f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:58.396684 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920ea88-8f25-43b1-81ef-aa362767016f admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:58.400167 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b920ea88-8f25-43b1-81ef-aa362767016f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:58.404754 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b920ea88-8f25-43b1-81ef-aa362767016f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:58.476144 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1091/2159] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:15:58 2022] POST /identity/v3/projects => generated 373 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:15:58.477338 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd84876-1482-4085-be25-82a463c22649 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:58.498495 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd84876-1482-4085-be25-82a463c22649 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:58.679650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb6ec31-3e99-4b3e-b94b-b98a8884d3f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:58.680444 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb6ec31-3e99-4b3e-b94b-b98a8884d3f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:58.680837 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb6ec31-3e99-4b3e-b94b-b98a8884d3f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:58.681251 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb6ec31-3e99-4b3e-b94b-b98a8884d3f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:58.683047 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb6ec31-3e99-4b3e-b94b-b98a8884d3f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:58.687406 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb6ec31-3e99-4b3e-b94b-b98a8884d3f0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:58.736402 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1069/2160] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:15:58 2022] POST /identity/v3/users => generated 293 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:15:58.740437 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b920ea88-8f25-43b1-81ef-aa362767016f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:58.758157 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b920ea88-8f25-43b1-81ef-aa362767016f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:58.777803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ace71-a5b5-4ee1-ae54-80bea66c735d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:58.778523 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ace71-a5b5-4ee1-ae54-80bea66c735d admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:58.781199 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ace71-a5b5-4ee1-ae54-80bea66c735d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:58.781503 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ace71-a5b5-4ee1-ae54-80bea66c735d admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:58.782269 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ace71-a5b5-4ee1-ae54-80bea66c735d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:58.784162 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ace71-a5b5-4ee1-ae54-80bea66c735d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:58.804011 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1092/2161] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:15:58.810444 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb6ec31-3e99-4b3e-b94b-b98a8884d3f0 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:58.829700 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb6ec31-3e99-4b3e-b94b-b98a8884d3f0 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:58.835056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f2f03-4d09-4f7e-82ab-a0ba7beb8b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:58.836395 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f2f03-4d09-4f7e-82ab-a0ba7beb8b25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:58.836918 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f2f03-4d09-4f7e-82ab-a0ba7beb8b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:58.837532 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8f2f03-4d09-4f7e-82ab-a0ba7beb8b25 admin admin] PATH_INFO: `/v3/projects/bb950dbf7b664b4da9681ef58335fa3b/users/e6d4d118ddbe40749ca7b0dbceb0f30c/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:58.848905 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1905aafd-bb60-40fe-9cdb-49b265b3e56d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:58.850498 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1905aafd-bb60-40fe-9cdb-49b265b3e56d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:58.851028 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1905aafd-bb60-40fe-9cdb-49b265b3e56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:58.891711 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8f2f03-4d09-4f7e-82ab-a0ba7beb8b25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb950dbf7b664b4da9681ef58335fa3b, user_id=e6d4d118ddbe40749ca7b0dbceb0f30c, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:58.895682 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1905aafd-bb60-40fe-9cdb-49b265b3e56d None None] MFA Rules not processed for user `9c9149cb2b3c423796164b487a305bbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:58.898576 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1905aafd-bb60-40fe-9cdb-49b265b3e56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:58.899580 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8f2f03-4d09-4f7e-82ab-a0ba7beb8b25 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:58.946514 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1070/2162] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:58 2022] PUT /identity/v3/projects/bb950dbf7b664b4da9681ef58335fa3b/users/e6d4d118ddbe40749ca7b0dbceb0f30c/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:58.949136 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f2f03-4d09-4f7e-82ab-a0ba7beb8b25 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:59.066087 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1093/2163] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:15:59.130264 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8f2f03-4d09-4f7e-82ab-a0ba7beb8b25 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:59.138633 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2e037f-a309-46a3-b582-6083898f2b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:59.139349 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e037f-a309-46a3-b582-6083898f2b44 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:59.139822 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e037f-a309-46a3-b582-6083898f2b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:59.140115 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2e037f-a309-46a3-b582-6083898f2b44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:59.140860 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2e037f-a309-46a3-b582-6083898f2b44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:59.142643 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2e037f-a309-46a3-b582-6083898f2b44 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:59.177444 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1071/2164] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:15:59.180644 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1905aafd-bb60-40fe-9cdb-49b265b3e56d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:59.197950 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1905aafd-bb60-40fe-9cdb-49b265b3e56d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:59.214241 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f134f1a-67f0-4d86-ae72-0ede4ef553e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:59.215953 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f134f1a-67f0-4d86-ae72-0ede4ef553e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:59.217339 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f134f1a-67f0-4d86-ae72-0ede4ef553e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:59.217841 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f134f1a-67f0-4d86-ae72-0ede4ef553e6 admin admin] PATH_INFO: `/v3/projects/bb950dbf7b664b4da9681ef58335fa3b/users/e6d4d118ddbe40749ca7b0dbceb0f30c/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:59.220590 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f134f1a-67f0-4d86-ae72-0ede4ef553e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb950dbf7b664b4da9681ef58335fa3b, user_id=e6d4d118ddbe40749ca7b0dbceb0f30c, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:59.235842 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f134f1a-67f0-4d86-ae72-0ede4ef553e6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:59.274282 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1094/2165] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:15:59 2022] PUT /identity/v3/projects/bb950dbf7b664b4da9681ef58335fa3b/users/e6d4d118ddbe40749ca7b0dbceb0f30c/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:15:59.277184 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910c2be-6c05-42f5-8804-6d3b80b8cfcf None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:59.277184 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910c2be-6c05-42f5-8804-6d3b80b8cfcf None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:59.277749 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5910c2be-6c05-42f5-8804-6d3b80b8cfcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:59.352086 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-5910c2be-6c05-42f5-8804-6d3b80b8cfcf None None] MFA Rules not processed for user `e6d4d118ddbe40749ca7b0dbceb0f30c`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:15:59.355474 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-5910c2be-6c05-42f5-8804-6d3b80b8cfcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:59.426927 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:15:59.463056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1072/2166] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:15:59 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:15:59.470973 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f134f1a-67f0-4d86-ae72-0ede4ef553e6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:15:59.472878 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3f134f1a-67f0-4d86-ae72-0ede4ef553e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:15:59.553082 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f134f1a-67f0-4d86-ae72-0ede4ef553e6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:15:59.556986 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f72dfa0-8bb5-4632-b804-eef4c8b17b24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:15:59.557646 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f72dfa0-8bb5-4632-b804-eef4c8b17b24 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:15:59.557927 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f72dfa0-8bb5-4632-b804-eef4c8b17b24 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:15:59.558199 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f72dfa0-8bb5-4632-b804-eef4c8b17b24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:15:59.569785 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f72dfa0-8bb5-4632-b804-eef4c8b17b24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:15:59.571410 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f72dfa0-8bb5-4632-b804-eef4c8b17b24 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:15:59.588517 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1095/2167] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:15:59 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:16:00.138780 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5910c2be-6c05-42f5-8804-6d3b80b8cfcf None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:00.276121 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5910c2be-6c05-42f5-8804-6d3b80b8cfcf None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:00.282191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa99837-246e-461d-8438-34544105948d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:00.283787 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa99837-246e-461d-8438-34544105948d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:00.284313 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa99837-246e-461d-8438-34544105948d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:00.285693 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa99837-246e-461d-8438-34544105948d admin admin] PATH_INFO: `/v3/users/c14700617be147a2b6a61df5e8c842f0` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:00.288301 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa99837-246e-461d-8438-34544105948d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c14700617be147a2b6a61df5e8c842f0)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:00.291786 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa99837-246e-461d-8438-34544105948d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:00.459779 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8aa99837-246e-461d-8438-34544105948d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c14700617be147a2b6a61df5e8c842f0'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:00.482833 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8aa99837-246e-461d-8438-34544105948d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c14700617be147a2b6a61df5e8c842f0'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:00.501434 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8aa99837-246e-461d-8438-34544105948d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c14700617be147a2b6a61df5e8c842f0'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:00.502376 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8aa99837-246e-461d-8438-34544105948d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c14700617be147a2b6a61df5e8c842f0'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:00.503184 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8aa99837-246e-461d-8438-34544105948d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c14700617be147a2b6a61df5e8c842f0'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:00.562504 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1073/2168] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:16:00 2022] DELETE /identity/v3/users/c14700617be147a2b6a61df5e8c842f0 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:16:00.565479 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd62fd8-1ba6-465e-8410-efcbbc35a967 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:00.566585 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd62fd8-1ba6-465e-8410-efcbbc35a967 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:00.567385 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd62fd8-1ba6-465e-8410-efcbbc35a967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:00.593617 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5fd62fd8-1ba6-465e-8410-efcbbc35a967 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:16:00.595173 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5fd62fd8-1ba6-465e-8410-efcbbc35a967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:00.635420 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:00.717677 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa99837-246e-461d-8438-34544105948d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:00.719054 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1096/2169] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:16:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:16:00.721241 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8aa99837-246e-461d-8438-34544105948d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:00.835653 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa99837-246e-461d-8438-34544105948d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:00.840588 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6388793d-cda6-47a1-a98b-e67ee767b730 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:00.841421 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388793d-cda6-47a1-a98b-e67ee767b730 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:00.841834 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388793d-cda6-47a1-a98b-e67ee767b730 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:00.842108 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6388793d-cda6-47a1-a98b-e67ee767b730 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:00.857412 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6388793d-cda6-47a1-a98b-e67ee767b730 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:00.858934 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6388793d-cda6-47a1-a98b-e67ee767b730 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:00.873175 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1074/2170] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:16:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:16:01.258746 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd62fd8-1ba6-465e-8410-efcbbc35a967 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:01.261316 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5fd62fd8-1ba6-465e-8410-efcbbc35a967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:01.279994 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd62fd8-1ba6-465e-8410-efcbbc35a967 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:01.284738 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a3608-06b9-4419-b8d9-7e88847d75d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:01.285638 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a3608-06b9-4419-b8d9-7e88847d75d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:01.285922 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a3608-06b9-4419-b8d9-7e88847d75d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:01.286199 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a3608-06b9-4419-b8d9-7e88847d75d6 admin admin] PATH_INFO: `/v3/projects/f1294c0637814f03880123420bf65b6f` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:01.287464 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56a3608-06b9-4419-b8d9-7e88847d75d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1294c0637814f03880123420bf65b6f)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:01.289417 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56a3608-06b9-4419-b8d9-7e88847d75d6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:01.375746 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a56a3608-06b9-4419-b8d9-7e88847d75d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1294c0637814f03880123420bf65b6f', 'request_id': 'req-a56a3608-06b9-4419-b8d9-7e88847d75d6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:01.386504 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a56a3608-06b9-4419-b8d9-7e88847d75d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1294c0637814f03880123420bf65b6f', 'request_id': 'req-a56a3608-06b9-4419-b8d9-7e88847d75d6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:01.419100 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a56a3608-06b9-4419-b8d9-7e88847d75d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1294c0637814f03880123420bf65b6f', 'request_id': 'req-a56a3608-06b9-4419-b8d9-7e88847d75d6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:01.438558 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a56a3608-06b9-4419-b8d9-7e88847d75d6 admin admin] The token cache is being invalidate because project f1294c0637814f03880123420bf65b6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:16:01.439141 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a56a3608-06b9-4419-b8d9-7e88847d75d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:01.442267 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a56a3608-06b9-4419-b8d9-7e88847d75d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:01.444450 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1097/2171] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:16:01 2022] DELETE /identity/v3/projects/f1294c0637814f03880123420bf65b6f => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:16:01.448915 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6388793d-cda6-47a1-a98b-e67ee767b730 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:01.466325 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6388793d-cda6-47a1-a98b-e67ee767b730 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:01.475847 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:01.513443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a3608-06b9-4419-b8d9-7e88847d75d6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:01.517966 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a56a3608-06b9-4419-b8d9-7e88847d75d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:01.663574 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85999d00-0e61-4379-96d5-0f50397e9988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:01.664542 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85999d00-0e61-4379-96d5-0f50397e9988 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:01.664829 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85999d00-0e61-4379-96d5-0f50397e9988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:01.665109 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85999d00-0e61-4379-96d5-0f50397e9988 admin admin] PATH_INFO: `/v3/users/e6d4d118ddbe40749ca7b0dbceb0f30c` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:01.671044 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85999d00-0e61-4379-96d5-0f50397e9988 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6d4d118ddbe40749ca7b0dbceb0f30c)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:01.673756 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85999d00-0e61-4379-96d5-0f50397e9988 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:01.722470 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a3608-06b9-4419-b8d9-7e88847d75d6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:01.736467 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d0d59-600d-4b4d-8f11-b95038f4e7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:01.738572 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d0d59-600d-4b4d-8f11-b95038f4e7a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:01.739547 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d0d59-600d-4b4d-8f11-b95038f4e7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:01.740111 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d0d59-600d-4b4d-8f11-b95038f4e7a9 admin admin] PATH_INFO: `/v3/users/7e3961e4d389465d9a3191408cbaaa5b` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:01.743051 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0d0d59-600d-4b4d-8f11-b95038f4e7a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e3961e4d389465d9a3191408cbaaa5b)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:01.748739 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0d0d59-600d-4b4d-8f11-b95038f4e7a9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:01.775069 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-85999d00-0e61-4379-96d5-0f50397e9988 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6d4d118ddbe40749ca7b0dbceb0f30c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:01.788483 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-85999d00-0e61-4379-96d5-0f50397e9988 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6d4d118ddbe40749ca7b0dbceb0f30c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:01.808167 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-85999d00-0e61-4379-96d5-0f50397e9988 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6d4d118ddbe40749ca7b0dbceb0f30c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:01.808938 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-85999d00-0e61-4379-96d5-0f50397e9988 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6d4d118ddbe40749ca7b0dbceb0f30c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:01.809669 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-85999d00-0e61-4379-96d5-0f50397e9988 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6d4d118ddbe40749ca7b0dbceb0f30c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:01.854431 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1075/2172] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:16:01 2022] DELETE /identity/v3/users/e6d4d118ddbe40749ca7b0dbceb0f30c => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:16:01.903484 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0c0d0d59-600d-4b4d-8f11-b95038f4e7a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e3961e4d389465d9a3191408cbaaa5b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:01.930670 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:01.936678 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0c0d0d59-600d-4b4d-8f11-b95038f4e7a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e3961e4d389465d9a3191408cbaaa5b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:01.939642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0c0d0d59-600d-4b4d-8f11-b95038f4e7a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e3961e4d389465d9a3191408cbaaa5b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:01.960841 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0c0d0d59-600d-4b4d-8f11-b95038f4e7a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e3961e4d389465d9a3191408cbaaa5b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:01.961917 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0c0d0d59-600d-4b4d-8f11-b95038f4e7a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e3961e4d389465d9a3191408cbaaa5b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:02.041873 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1098/2173] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:16:01 2022] DELETE /identity/v3/users/7e3961e4d389465d9a3191408cbaaa5b => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:16:02.046723 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e735e4-9286-46a5-b231-6242c3aa97bc None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:02.047046 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e735e4-9286-46a5-b231-6242c3aa97bc None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:02.047325 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e735e4-9286-46a5-b231-6242c3aa97bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:02.088908 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e3e735e4-9286-46a5-b231-6242c3aa97bc None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:16:02.090379 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e3e735e4-9286-46a5-b231-6242c3aa97bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:02.180458 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1076/2174] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:16:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:16:02.192712 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d0d59-600d-4b4d-8f11-b95038f4e7a9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:02.194507 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0c0d0d59-600d-4b4d-8f11-b95038f4e7a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:02.288978 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d0d59-600d-4b4d-8f11-b95038f4e7a9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:02.294555 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df78c0e-cff8-4a21-bfef-b4f4bd5447ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:02.295550 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df78c0e-cff8-4a21-bfef-b4f4bd5447ba service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:02.295844 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df78c0e-cff8-4a21-bfef-b4f4bd5447ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:02.296127 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df78c0e-cff8-4a21-bfef-b4f4bd5447ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:02.308285 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df78c0e-cff8-4a21-bfef-b4f4bd5447ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:02.310199 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df78c0e-cff8-4a21-bfef-b4f4bd5447ba service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:02.323074 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1099/2175] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:16:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:16:02.511801 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e735e4-9286-46a5-b231-6242c3aa97bc None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:02.513374 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e3e735e4-9286-46a5-b231-6242c3aa97bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:02.531259 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e735e4-9286-46a5-b231-6242c3aa97bc None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:02.555747 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b56b686-2229-4222-b31d-5e30e2cb9627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:02.559527 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56b686-2229-4222-b31d-5e30e2cb9627 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:02.559930 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56b686-2229-4222-b31d-5e30e2cb9627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:02.560339 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b56b686-2229-4222-b31d-5e30e2cb9627 admin admin] PATH_INFO: `/v3/projects/bb950dbf7b664b4da9681ef58335fa3b` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:02.564545 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b56b686-2229-4222-b31d-5e30e2cb9627 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb950dbf7b664b4da9681ef58335fa3b)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:02.566754 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b56b686-2229-4222-b31d-5e30e2cb9627 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:02.652051 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:02.707559 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9b56b686-2229-4222-b31d-5e30e2cb9627 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb950dbf7b664b4da9681ef58335fa3b', 'request_id': 'req-9b56b686-2229-4222-b31d-5e30e2cb9627'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:02.722304 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9b56b686-2229-4222-b31d-5e30e2cb9627 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb950dbf7b664b4da9681ef58335fa3b', 'request_id': 'req-9b56b686-2229-4222-b31d-5e30e2cb9627'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:02.744875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9b56b686-2229-4222-b31d-5e30e2cb9627 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb950dbf7b664b4da9681ef58335fa3b', 'request_id': 'req-9b56b686-2229-4222-b31d-5e30e2cb9627'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:02.766490 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df78c0e-cff8-4a21-bfef-b4f4bd5447ba service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:02.769087 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2df78c0e-cff8-4a21-bfef-b4f4bd5447ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:02.771766 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9b56b686-2229-4222-b31d-5e30e2cb9627 admin admin] The token cache is being invalidate because project bb950dbf7b664b4da9681ef58335fa3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:16:02.772065 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9b56b686-2229-4222-b31d-5e30e2cb9627 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:02.773503 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9b56b686-2229-4222-b31d-5e30e2cb9627 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:02.776652 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1077/2176] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:16:02 2022] DELETE /identity/v3/projects/bb950dbf7b664b4da9681ef58335fa3b => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:16:02.782075 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804551cd-d523-4cde-b285-38392fbcaf23 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:02.782257 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804551cd-d523-4cde-b285-38392fbcaf23 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:02.782502 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804551cd-d523-4cde-b285-38392fbcaf23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:02.821577 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df78c0e-cff8-4a21-bfef-b4f4bd5447ba service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:02.824547 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-804551cd-d523-4cde-b285-38392fbcaf23 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:16:02.827866 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-804551cd-d523-4cde-b285-38392fbcaf23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:02.932281 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1078/2177] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:16:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:16:02.938880 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c496c2-eb44-4c37-a06d-c3707d10f935 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:02.939343 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c496c2-eb44-4c37-a06d-c3707d10f935 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:02.939619 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c496c2-eb44-4c37-a06d-c3707d10f935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:02.965291 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d336c-41b2-4016-a963-a31ee205a672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:02.966832 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d336c-41b2-4016-a963-a31ee205a672 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:02.966832 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d336c-41b2-4016-a963-a31ee205a672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:02.966832 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d336c-41b2-4016-a963-a31ee205a672 admin admin] PATH_INFO: `/v3/projects/5807ebd6499d40cf8461813e1006e7d6` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:02.969077 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d336c-41b2-4016-a963-a31ee205a672 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5807ebd6499d40cf8461813e1006e7d6)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:02.972204 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d336c-41b2-4016-a963-a31ee205a672 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:02.981135 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-68c496c2-eb44-4c37-a06d-c3707d10f935 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:16:02.982891 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-68c496c2-eb44-4c37-a06d-c3707d10f935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:03.008807 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1079/2178] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:16:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:16:03.013630 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c496c2-eb44-4c37-a06d-c3707d10f935 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:03.033294 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c496c2-eb44-4c37-a06d-c3707d10f935 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:03.043239 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a786e4d-4823-4d70-b0ef-799bca7fc548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:03.043931 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a786e4d-4823-4d70-b0ef-799bca7fc548 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:03.044248 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a786e4d-4823-4d70-b0ef-799bca7fc548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:03.044516 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a786e4d-4823-4d70-b0ef-799bca7fc548 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:03.045248 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a786e4d-4823-4d70-b0ef-799bca7fc548 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:16:03.045554 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a786e4d-4823-4d70-b0ef-799bca7fc548 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:03.050782 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a786e4d-4823-4d70-b0ef-799bca7fc548 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:03.081119 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1080/2179] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:16:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:16:03.082003 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-006d336c-41b2-4016-a963-a31ee205a672 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5807ebd6499d40cf8461813e1006e7d6', 'request_id': 'req-006d336c-41b2-4016-a963-a31ee205a672'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:03.084096 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a786e4d-4823-4d70-b0ef-799bca7fc548 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:03.097719 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-006d336c-41b2-4016-a963-a31ee205a672 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5807ebd6499d40cf8461813e1006e7d6', 'request_id': 'req-006d336c-41b2-4016-a963-a31ee205a672'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:03.108755 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-006d336c-41b2-4016-a963-a31ee205a672 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5807ebd6499d40cf8461813e1006e7d6', 'request_id': 'req-006d336c-41b2-4016-a963-a31ee205a672'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:03.110177 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a786e4d-4823-4d70-b0ef-799bca7fc548 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:03.131730 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-006d336c-41b2-4016-a963-a31ee205a672 admin admin] The token cache is being invalidate because project 5807ebd6499d40cf8461813e1006e7d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:16:03.132022 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-006d336c-41b2-4016-a963-a31ee205a672 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:03.132996 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-006d336c-41b2-4016-a963-a31ee205a672 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:03.142391 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1100/2180] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:16:02 2022] DELETE /identity/v3/projects/5807ebd6499d40cf8461813e1006e7d6 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:16:03.143882 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d5c4f0-5590-46f6-acc2-04e8ea72a3f3 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:03.144165 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d5c4f0-5590-46f6-acc2-04e8ea72a3f3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:03.144437 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d5c4f0-5590-46f6-acc2-04e8ea72a3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:03.163311 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:03.193675 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-47d5c4f0-5590-46f6-acc2-04e8ea72a3f3 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:16:03.194903 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-47d5c4f0-5590-46f6-acc2-04e8ea72a3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:03.280415 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c7059-37d1-4a12-a2a4-aa41bc44bcf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:03.281190 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c7059-37d1-4a12-a2a4-aa41bc44bcf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:03.281539 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c7059-37d1-4a12-a2a4-aa41bc44bcf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:03.287883 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c7059-37d1-4a12-a2a4-aa41bc44bcf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:03.288728 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29c7059-37d1-4a12-a2a4-aa41bc44bcf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:03.295235 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29c7059-37d1-4a12-a2a4-aa41bc44bcf5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:03.390719 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1081/2181] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:16:03 2022] POST /identity/v3/projects => generated 385 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:16:03.391685 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c7059-37d1-4a12-a2a4-aa41bc44bcf5 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:03.394008 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d29c7059-37d1-4a12-a2a4-aa41bc44bcf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:03.416337 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:03.428793 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1101/2182] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:16:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:16:03.438063 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3f34c-7fe0-4961-ac52-82e7401fc105 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:03.438363 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3f34c-7fe0-4961-ac52-82e7401fc105 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:03.438646 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3f34c-7fe0-4961-ac52-82e7401fc105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:03.476780 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4be3f34c-7fe0-4961-ac52-82e7401fc105 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:16:03.478921 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4be3f34c-7fe0-4961-ac52-82e7401fc105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:03.538391 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c7059-37d1-4a12-a2a4-aa41bc44bcf5 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:03.552856 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97c756b-b1e9-4d61-8a47-9f3fe9bec6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:03.553522 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c756b-b1e9-4d61-8a47-9f3fe9bec6de admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:03.553805 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c756b-b1e9-4d61-8a47-9f3fe9bec6de admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:03.554078 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97c756b-b1e9-4d61-8a47-9f3fe9bec6de admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:03.554818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97c756b-b1e9-4d61-8a47-9f3fe9bec6de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:03.557363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97c756b-b1e9-4d61-8a47-9f3fe9bec6de admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:03.584004 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1082/2183] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:16:03 2022] POST /identity/v3/users => generated 293 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:16:03.589942 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97c756b-b1e9-4d61-8a47-9f3fe9bec6de admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:03.594600 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1102/2184] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:16:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:16:03.602910 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be3f34c-7fe0-4961-ac52-82e7401fc105 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:03.612033 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97c756b-b1e9-4d61-8a47-9f3fe9bec6de admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:03.622235 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be3f34c-7fe0-4961-ac52-82e7401fc105 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:03.627151 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b6ea5e-3dc5-42f9-aa92-16317f187168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:03.627956 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6ea5e-3dc5-42f9-aa92-16317f187168 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:03.628188 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6ea5e-3dc5-42f9-aa92-16317f187168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:03.628566 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6ea5e-3dc5-42f9-aa92-16317f187168 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:03.630148 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b6ea5e-3dc5-42f9-aa92-16317f187168 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:03.634074 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b6ea5e-3dc5-42f9-aa92-16317f187168 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:03.637071 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23235117-3afe-4ddd-b8c4-d98fe747f901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:03.637752 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23235117-3afe-4ddd-b8c4-d98fe747f901 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:03.638037 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23235117-3afe-4ddd-b8c4-d98fe747f901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:03.638309 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23235117-3afe-4ddd-b8c4-d98fe747f901 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:03.639783 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23235117-3afe-4ddd-b8c4-d98fe747f901 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:16:03.640093 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23235117-3afe-4ddd-b8c4-d98fe747f901 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:03.643323 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23235117-3afe-4ddd-b8c4-d98fe747f901 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:03.659728 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1083/2185] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:16:03.663931 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b6ea5e-3dc5-42f9-aa92-16317f187168 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:03.680263 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1103/2186] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:16:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:16:03.680812 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b6ea5e-3dc5-42f9-aa92-16317f187168 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:03.681206 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23235117-3afe-4ddd-b8c4-d98fe747f901 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:03.692452 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f08c9bb-e30d-434c-bddd-e5ff7ec5dc90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:03.693410 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08c9bb-e30d-434c-bddd-e5ff7ec5dc90 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:03.693701 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08c9bb-e30d-434c-bddd-e5ff7ec5dc90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:03.695194 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08c9bb-e30d-434c-bddd-e5ff7ec5dc90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:03.695959 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f08c9bb-e30d-434c-bddd-e5ff7ec5dc90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:03.698722 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f08c9bb-e30d-434c-bddd-e5ff7ec5dc90 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:03.717590 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1084/2187] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:16:03.722151 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23235117-3afe-4ddd-b8c4-d98fe747f901 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:03.724835 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f08c9bb-e30d-434c-bddd-e5ff7ec5dc90 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:03.736186 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b005b72e-951b-4797-bebe-abda198ab717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:03.737151 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b005b72e-951b-4797-bebe-abda198ab717 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:03.737616 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b005b72e-951b-4797-bebe-abda198ab717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:03.738092 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b005b72e-951b-4797-bebe-abda198ab717 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:03.739095 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b005b72e-951b-4797-bebe-abda198ab717 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:03.741233 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b005b72e-951b-4797-bebe-abda198ab717 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:03.750669 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f08c9bb-e30d-434c-bddd-e5ff7ec5dc90 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:03.760694 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b46a8e3-60f9-4125-beb4-873b20fb3ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:03.761662 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46a8e3-60f9-4125-beb4-873b20fb3ba5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:03.761910 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46a8e3-60f9-4125-beb4-873b20fb3ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:03.762189 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46a8e3-60f9-4125-beb4-873b20fb3ba5 admin admin] PATH_INFO: `/v3/projects/ec4c9c703987400ca2228a76cfe52f75/users/0f213311906f4b859431ef75f86c9629/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:03.779681 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1104/2188] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:16:03 2022] POST /identity/v3/projects => generated 383 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:16:03.781263 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b005b72e-951b-4797-bebe-abda198ab717 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:03.804214 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b46a8e3-60f9-4125-beb4-873b20fb3ba5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec4c9c703987400ca2228a76cfe52f75, user_id=0f213311906f4b859431ef75f86c9629, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:03.808955 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b46a8e3-60f9-4125-beb4-873b20fb3ba5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:03.864025 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1085/2189] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:16:03 2022] PUT /identity/v3/projects/ec4c9c703987400ca2228a76cfe52f75/users/0f213311906f4b859431ef75f86c9629/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:16:03.877424 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db6326-c09d-42e0-82d4-c50d347b6205 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:03.877726 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db6326-c09d-42e0-82d4-c50d347b6205 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:03.877990 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db6326-c09d-42e0-82d4-c50d347b6205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:03.879460 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b005b72e-951b-4797-bebe-abda198ab717 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:03.897677 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:03.954975 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-19db6326-c09d-42e0-82d4-c50d347b6205 None None] MFA Rules not processed for user `0f213311906f4b859431ef75f86c9629`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:16:03.956596 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-19db6326-c09d-42e0-82d4-c50d347b6205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:03.960643 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f68ccb2-8019-48db-a7d1-535a3768ee27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:03.961340 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68ccb2-8019-48db-a7d1-535a3768ee27 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:03.961623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68ccb2-8019-48db-a7d1-535a3768ee27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:03.961898 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f68ccb2-8019-48db-a7d1-535a3768ee27 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:03.962781 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f68ccb2-8019-48db-a7d1-535a3768ee27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:03.965115 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f68ccb2-8019-48db-a7d1-535a3768ee27 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:03.997802 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1105/2190] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:16:03 2022] POST /identity/v3/users => generated 292 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:16:03.998998 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f68ccb2-8019-48db-a7d1-535a3768ee27 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:04.012420 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f68ccb2-8019-48db-a7d1-535a3768ee27 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:04.021495 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b51a14-6ec2-4f11-b7b7-4ed4110b27c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:04.022184 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b51a14-6ec2-4f11-b7b7-4ed4110b27c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:04.022473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b51a14-6ec2-4f11-b7b7-4ed4110b27c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:04.022748 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b51a14-6ec2-4f11-b7b7-4ed4110b27c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:04.023521 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b51a14-6ec2-4f11-b7b7-4ed4110b27c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:04.025433 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b51a14-6ec2-4f11-b7b7-4ed4110b27c7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:04.045426 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1106/2191] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:16:04.047126 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b51a14-6ec2-4f11-b7b7-4ed4110b27c7 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:04.062383 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b51a14-6ec2-4f11-b7b7-4ed4110b27c7 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:04.070075 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82564a72-71c0-43f6-9c64-c3ea9b06cfb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:04.071561 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82564a72-71c0-43f6-9c64-c3ea9b06cfb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:04.073468 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82564a72-71c0-43f6-9c64-c3ea9b06cfb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:04.074037 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82564a72-71c0-43f6-9c64-c3ea9b06cfb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:04.075063 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82564a72-71c0-43f6-9c64-c3ea9b06cfb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:04.078627 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82564a72-71c0-43f6-9c64-c3ea9b06cfb8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:04.100313 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1107/2192] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:16:04 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:16:04.101537 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82564a72-71c0-43f6-9c64-c3ea9b06cfb8 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:04.135024 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1086/2193] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:16:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:16:04.137857 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82564a72-71c0-43f6-9c64-c3ea9b06cfb8 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:04.139052 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db6326-c09d-42e0-82d4-c50d347b6205 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:04.143914 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ec6768-64c0-4ff4-9bbf-8d933fdc3e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:04.146354 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec6768-64c0-4ff4-9bbf-8d933fdc3e34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:04.147002 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec6768-64c0-4ff4-9bbf-8d933fdc3e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:04.147546 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec6768-64c0-4ff4-9bbf-8d933fdc3e34 admin admin] PATH_INFO: `/v3/projects/cb7a685d4f744bb98d1b3063d03db0e7/users/081d618f848e4a119df5954927a6371e/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:04.168404 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db6326-c09d-42e0-82d4-c50d347b6205 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:04.169396 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ec6768-64c0-4ff4-9bbf-8d933fdc3e34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb7a685d4f744bb98d1b3063d03db0e7, user_id=081d618f848e4a119df5954927a6371e, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:04.172797 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ec6768-64c0-4ff4-9bbf-8d933fdc3e34 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:04.181878 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed67a872-bb83-40b4-be9c-dab5c3398a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:04.212358 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67a872-bb83-40b4-be9c-dab5c3398a96 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:04.212358 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67a872-bb83-40b4-be9c-dab5c3398a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:04.212358 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67a872-bb83-40b4-be9c-dab5c3398a96 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:04.212358 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed67a872-bb83-40b4-be9c-dab5c3398a96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:04.213676 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1108/2194] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:16:04 2022] PUT /identity/v3/projects/cb7a685d4f744bb98d1b3063d03db0e7/users/081d618f848e4a119df5954927a6371e/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:16:04.214101 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed67a872-bb83-40b4-be9c-dab5c3398a96 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:04.218751 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0e0e9-9682-4b26-97aa-38154d44c28b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:04.220806 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0e0e9-9682-4b26-97aa-38154d44c28b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:04.221316 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb0e0e9-9682-4b26-97aa-38154d44c28b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:04.262995 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6bb0e0e9-9682-4b26-97aa-38154d44c28b None None] MFA Rules not processed for user `081d618f848e4a119df5954927a6371e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:16:04.264402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6bb0e0e9-9682-4b26-97aa-38154d44c28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:04.289412 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1087/2195] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:16:04 2022] POST /identity/v3/projects => generated 385 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:16:04.293978 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed67a872-bb83-40b4-be9c-dab5c3398a96 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:04.354729 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:04.401965 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1109/2196] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:16:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:16:04.419645 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0e0e9-9682-4b26-97aa-38154d44c28b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:04.454156 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed67a872-bb83-40b4-be9c-dab5c3398a96 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:04.472713 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0a6d8f-2756-4354-9465-1af9cb7f5a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:04.473413 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a6d8f-2756-4354-9465-1af9cb7f5a29 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:04.478946 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a6d8f-2756-4354-9465-1af9cb7f5a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:04.479223 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a6d8f-2756-4354-9465-1af9cb7f5a29 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:04.490053 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0a6d8f-2756-4354-9465-1af9cb7f5a29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:04.491972 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0a6d8f-2756-4354-9465-1af9cb7f5a29 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:04.503226 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:04.534474 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb0e0e9-9682-4b26-97aa-38154d44c28b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:04.547119 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ed6ac3-b671-451d-b4b8-68cdf3848f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:04.547119 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed6ac3-b671-451d-b4b8-68cdf3848f5b admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:04.547119 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed6ac3-b671-451d-b4b8-68cdf3848f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:04.547119 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed6ac3-b671-451d-b4b8-68cdf3848f5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:04.547466 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ed6ac3-b671-451d-b4b8-68cdf3848f5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:04.547466 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ed6ac3-b671-451d-b4b8-68cdf3848f5b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:04.568202 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1088/2197] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:16:04 2022] POST /identity/v3/users => generated 299 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:16:04.572403 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0a6d8f-2756-4354-9465-1af9cb7f5a29 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:04.589259 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1110/2198] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:16:04 2022] POST /identity/v3/projects => generated 383 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:16:04.591033 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0a6d8f-2756-4354-9465-1af9cb7f5a29 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:04.591686 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ed6ac3-b671-451d-b4b8-68cdf3848f5b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:04.750613 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c358936-0aba-497d-aefa-a899fe5e0308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:04.751641 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c358936-0aba-497d-aefa-a899fe5e0308 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:04.751762 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ed6ac3-b671-451d-b4b8-68cdf3848f5b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:04.753888 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c358936-0aba-497d-aefa-a899fe5e0308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:04.754279 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c358936-0aba-497d-aefa-a899fe5e0308 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:04.755228 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c358936-0aba-497d-aefa-a899fe5e0308 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:04.759157 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c358936-0aba-497d-aefa-a899fe5e0308 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:04.765782 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ebef43-9b48-4386-9269-42b97f02d5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:04.766601 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebef43-9b48-4386-9269-42b97f02d5f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:04.766917 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebef43-9b48-4386-9269-42b97f02d5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:04.767199 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebef43-9b48-4386-9269-42b97f02d5f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:04.768330 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ebef43-9b48-4386-9269-42b97f02d5f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:04.776439 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ebef43-9b48-4386-9269-42b97f02d5f5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:04.783154 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1089/2199] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:16:04 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:16:04.786993 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c358936-0aba-497d-aefa-a899fe5e0308 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:04.801444 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c358936-0aba-497d-aefa-a899fe5e0308 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:04.822340 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc11ed-b1f5-428e-82d5-588fb73cd6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:04.822831 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1111/2200] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:16:04 2022] POST /identity/v3/users => generated 292 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:16:04.824014 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ebef43-9b48-4386-9269-42b97f02d5f5 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:04.825063 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc11ed-b1f5-428e-82d5-588fb73cd6ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:04.825347 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc11ed-b1f5-428e-82d5-588fb73cd6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:04.826811 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc11ed-b1f5-428e-82d5-588fb73cd6ab admin admin] PATH_INFO: `/v3/projects/7349e8090d1c450d8e7d6fa2f7b9bab3/users/f6133bf47e974848a2a05f8cee26f52d/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:04.881760 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ebef43-9b48-4386-9269-42b97f02d5f5 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:04.915142 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0028eb02-e495-4a6f-9946-e447bbbc37f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:04.915142 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028eb02-e495-4a6f-9946-e447bbbc37f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:04.915142 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028eb02-e495-4a6f-9946-e447bbbc37f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:04.915587 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028eb02-e495-4a6f-9946-e447bbbc37f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:04.916585 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0028eb02-e495-4a6f-9946-e447bbbc37f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:04.918654 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcc11ed-b1f5-428e-82d5-588fb73cd6ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=7349e8090d1c450d8e7d6fa2f7b9bab3, user_id=f6133bf47e974848a2a05f8cee26f52d, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:04.921631 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcc11ed-b1f5-428e-82d5-588fb73cd6ab admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:04.925549 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0028eb02-e495-4a6f-9946-e447bbbc37f8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:04.954608 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1112/2201] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:16:04 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:16:04.960589 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1090/2202] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:16:04 2022] PUT /identity/v3/projects/7349e8090d1c450d8e7d6fa2f7b9bab3/users/f6133bf47e974848a2a05f8cee26f52d/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:16:04.961676 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0028eb02-e495-4a6f-9946-e447bbbc37f8 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:04.964355 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc11ed-b1f5-428e-82d5-588fb73cd6ab admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:04.994602 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0028eb02-e495-4a6f-9946-e447bbbc37f8 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:05.122678 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc11ed-b1f5-428e-82d5-588fb73cd6ab admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:05.146472 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbd6176-bbb9-4ba2-8477-c79ef156d38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:05.147669 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd6176-bbb9-4ba2-8477-c79ef156d38f admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:05.148109 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd6176-bbb9-4ba2-8477-c79ef156d38f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:05.148532 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd6176-bbb9-4ba2-8477-c79ef156d38f admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:05.149421 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbd6176-bbb9-4ba2-8477-c79ef156d38f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:05.152117 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbd6176-bbb9-4ba2-8477-c79ef156d38f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:05.176408 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b68a8d1-c559-4429-958d-04698bf86955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:05.177106 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b68a8d1-c559-4429-958d-04698bf86955 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:05.177391 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b68a8d1-c559-4429-958d-04698bf86955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:05.177669 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b68a8d1-c559-4429-958d-04698bf86955 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:05.178208 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1091/2203] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:16:04 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:16:05.178497 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b68a8d1-c559-4429-958d-04698bf86955 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:05.180155 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b68a8d1-c559-4429-958d-04698bf86955 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:05.183495 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbd6176-bbb9-4ba2-8477-c79ef156d38f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:05.197458 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbd6176-bbb9-4ba2-8477-c79ef156d38f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:05.204430 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41539366-4e2e-4f4c-a5da-8a1af1192a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:05.206767 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41539366-4e2e-4f4c-a5da-8a1af1192a6e admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:05.207285 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41539366-4e2e-4f4c-a5da-8a1af1192a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:05.207756 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41539366-4e2e-4f4c-a5da-8a1af1192a6e admin admin] PATH_INFO: `/v3/projects/7349e8090d1c450d8e7d6fa2f7b9bab3/users/f6133bf47e974848a2a05f8cee26f52d/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:05.210150 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41539366-4e2e-4f4c-a5da-8a1af1192a6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7349e8090d1c450d8e7d6fa2f7b9bab3, user_id=f6133bf47e974848a2a05f8cee26f52d, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:05.214320 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41539366-4e2e-4f4c-a5da-8a1af1192a6e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:05.219213 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1113/2204] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:16:04 2022] GET /identity/v3/roles => generated 1420 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:16:05.223091 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b68a8d1-c559-4429-958d-04698bf86955 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:05.243263 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b68a8d1-c559-4429-958d-04698bf86955 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:05.305618 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1092/2205] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:16:05 2022] PUT /identity/v3/projects/7349e8090d1c450d8e7d6fa2f7b9bab3/users/f6133bf47e974848a2a05f8cee26f52d/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:16:05.308663 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1ec94-183f-4a83-961c-d361425f1464 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:05.308957 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1ec94-183f-4a83-961c-d361425f1464 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:05.309244 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1ec94-183f-4a83-961c-d361425f1464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:05.341761 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:05.371142 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-03a1ec94-183f-4a83-961c-d361425f1464 None None] MFA Rules not processed for user `f6133bf47e974848a2a05f8cee26f52d`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:16:05.373009 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-03a1ec94-183f-4a83-961c-d361425f1464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:05.394513 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:05.423313 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95281f0-ff58-476c-8f7f-0061882e7799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:05.424239 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95281f0-ff58-476c-8f7f-0061882e7799 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:05.424523 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95281f0-ff58-476c-8f7f-0061882e7799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:05.424801 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95281f0-ff58-476c-8f7f-0061882e7799 admin admin] PATH_INFO: `/v3/projects/12ad6a8f9246427c8a4f146ec172cd1e/users/9f4f2d95083544a78dc599ed6849e05e/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:05.481333 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95281f0-ff58-476c-8f7f-0061882e7799 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12ad6a8f9246427c8a4f146ec172cd1e, user_id=9f4f2d95083544a78dc599ed6849e05e, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:05.484746 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95281f0-ff58-476c-8f7f-0061882e7799 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:05.518178 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1114/2206] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:16:05 2022] PUT /identity/v3/projects/12ad6a8f9246427c8a4f146ec172cd1e/users/9f4f2d95083544a78dc599ed6849e05e/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:16:05.529142 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf29410-0fd5-4412-b093-58ac96362dc2 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:05.530274 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf29410-0fd5-4412-b093-58ac96362dc2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:05.530274 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf29410-0fd5-4412-b093-58ac96362dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:05.565320 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1093/2207] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:16:05 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:16:05.568516 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a1ec94-183f-4a83-961c-d361425f1464 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:05.576246 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-03a1ec94-183f-4a83-961c-d361425f1464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:05.602998 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-bcf29410-0fd5-4412-b093-58ac96362dc2 None None] MFA Rules not processed for user `9f4f2d95083544a78dc599ed6849e05e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:16:05.604297 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bcf29410-0fd5-4412-b093-58ac96362dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:05.773789 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a1ec94-183f-4a83-961c-d361425f1464 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:05.832583 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1115/2208] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:16:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:16:05.859933 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf29410-0fd5-4412-b093-58ac96362dc2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:05.894447 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea71d9e-53e3-40e7-948b-3f0160127385 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:05.894447 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea71d9e-53e3-40e7-948b-3f0160127385 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:05.894675 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea71d9e-53e3-40e7-948b-3f0160127385 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:05.894777 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea71d9e-53e3-40e7-948b-3f0160127385 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:06.016555 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea71d9e-53e3-40e7-948b-3f0160127385 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:06.018042 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea71d9e-53e3-40e7-948b-3f0160127385 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:06.057030 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1094/2209] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:16:05 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:16:06.080090 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf29410-0fd5-4412-b093-58ac96362dc2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:06.087992 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcfe2c5-0eef-4ac3-ab77-f4aacf303627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:06.088477 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcfe2c5-0eef-4ac3-ab77-f4aacf303627 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:06.088766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcfe2c5-0eef-4ac3-ab77-f4aacf303627 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:06.089232 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcfe2c5-0eef-4ac3-ab77-f4aacf303627 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:06.089898 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcfe2c5-0eef-4ac3-ab77-f4aacf303627 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:06.098222 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcfe2c5-0eef-4ac3-ab77-f4aacf303627 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:06.130727 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea71d9e-53e3-40e7-948b-3f0160127385 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:06.135877 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:06.138299 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1116/2210] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:16:05 2022] POST /identity/v3/projects => generated 381 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:16:06.143661 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcfe2c5-0eef-4ac3-ab77-f4aacf303627 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:06.172903 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea71d9e-53e3-40e7-948b-3f0160127385 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:06.241943 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:06.271523 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcfe2c5-0eef-4ac3-ab77-f4aacf303627 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:06.278791 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9085d11a-2695-4004-b925-9c25419c3ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:06.279824 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9085d11a-2695-4004-b925-9c25419c3ef6 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:06.279824 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9085d11a-2695-4004-b925-9c25419c3ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:06.280061 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9085d11a-2695-4004-b925-9c25419c3ef6 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:06.280759 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9085d11a-2695-4004-b925-9c25419c3ef6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:06.288791 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9085d11a-2695-4004-b925-9c25419c3ef6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:06.321091 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a75110-bdd0-476d-b93b-75813e2b1d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:06.323147 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a75110-bdd0-476d-b93b-75813e2b1d4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:06.325112 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a75110-bdd0-476d-b93b-75813e2b1d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:06.326608 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a75110-bdd0-476d-b93b-75813e2b1d4a admin admin] PATH_INFO: `/v3/users/0f213311906f4b859431ef75f86c9629` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:06.333128 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a75110-bdd0-476d-b93b-75813e2b1d4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f213311906f4b859431ef75f86c9629)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:06.335976 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a75110-bdd0-476d-b93b-75813e2b1d4a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:06.368953 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1117/2211] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:16:06 2022] POST /identity/v3/users => generated 297 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:16:06.374035 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9085d11a-2695-4004-b925-9c25419c3ef6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:06.409980 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9085d11a-2695-4004-b925-9c25419c3ef6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:06.431564 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38359e02-feec-4664-9f27-d4184b394901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:06.435268 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38359e02-feec-4664-9f27-d4184b394901 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:06.438077 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38359e02-feec-4664-9f27-d4184b394901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:06.438947 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38359e02-feec-4664-9f27-d4184b394901 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:06.443313 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38359e02-feec-4664-9f27-d4184b394901 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:06.448762 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38359e02-feec-4664-9f27-d4184b394901 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:06.472864 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b7a75110-bdd0-476d-b93b-75813e2b1d4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f213311906f4b859431ef75f86c9629'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:06.476146 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1118/2212] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:16:06 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:16:06.481003 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38359e02-feec-4664-9f27-d4184b394901 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:06.489624 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b7a75110-bdd0-476d-b93b-75813e2b1d4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f213311906f4b859431ef75f86c9629'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:06.504627 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38359e02-feec-4664-9f27-d4184b394901 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:06.516263 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b7a75110-bdd0-476d-b93b-75813e2b1d4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f213311906f4b859431ef75f86c9629'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:06.517845 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b7a75110-bdd0-476d-b93b-75813e2b1d4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f213311906f4b859431ef75f86c9629'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:06.518122 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b7a75110-bdd0-476d-b93b-75813e2b1d4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f213311906f4b859431ef75f86c9629'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:06.525397 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa231869-a595-498a-9364-8bd6aed3258a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:06.528000 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa231869-a595-498a-9364-8bd6aed3258a admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:06.528414 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa231869-a595-498a-9364-8bd6aed3258a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:06.528605 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa231869-a595-498a-9364-8bd6aed3258a admin admin] PATH_INFO: `/v3/projects/6782c945d7464b73bf482adaff579710/users/7ea245573911407e92242319231f1c27/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:06.579098 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1095/2213] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:16:06 2022] DELETE /identity/v3/users/0f213311906f4b859431ef75f86c9629 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:16:06.581244 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9836a1c-3fce-4b40-b764-f4c26444d4d8 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:06.581798 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9836a1c-3fce-4b40-b764-f4c26444d4d8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:06.582875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9836a1c-3fce-4b40-b764-f4c26444d4d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:06.585092 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa231869-a595-498a-9364-8bd6aed3258a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6782c945d7464b73bf482adaff579710, user_id=7ea245573911407e92242319231f1c27, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:06.588858 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa231869-a595-498a-9364-8bd6aed3258a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:06.618493 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1119/2214] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:16:06 2022] PUT /identity/v3/projects/6782c945d7464b73bf482adaff579710/users/7ea245573911407e92242319231f1c27/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:16:06.620078 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fcf720-61b2-436f-9e58-02900ce99750 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:06.622920 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fcf720-61b2-436f-9e58-02900ce99750 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:06.624247 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fcf720-61b2-436f-9e58-02900ce99750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:06.653478 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a9836a1c-3fce-4b40-b764-f4c26444d4d8 None None] MFA Rules not processed for user `18ff5bfbc9f8453b981fb228ca2c1b6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:16:06.656528 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a9836a1c-3fce-4b40-b764-f4c26444d4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:06.673185 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-82fcf720-61b2-436f-9e58-02900ce99750 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:16:06.677210 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-82fcf720-61b2-436f-9e58-02900ce99750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:06.859115 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1120/2215] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:16:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:16:06.861994 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fcf720-61b2-436f-9e58-02900ce99750 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:06.862243 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1096/2216] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:16:06 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:16:06.869389 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-82fcf720-61b2-436f-9e58-02900ce99750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:06.872011 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9836a1c-3fce-4b40-b764-f4c26444d4d8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:06.874400 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a9836a1c-3fce-4b40-b764-f4c26444d4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:06.912855 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:06.913297 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fcf720-61b2-436f-9e58-02900ce99750 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:06.923032 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bbbfe0-26a8-4be5-b5c3-376bc102c4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:06.924181 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bbbfe0-26a8-4be5-b5c3-376bc102c4c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:06.924181 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bbbfe0-26a8-4be5-b5c3-376bc102c4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:06.924916 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bbbfe0-26a8-4be5-b5c3-376bc102c4c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:06.927719 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bbbfe0-26a8-4be5-b5c3-376bc102c4c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:06.939464 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bbbfe0-26a8-4be5-b5c3-376bc102c4c1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:06.973725 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1121/2217] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:16:06 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:16:06.976998 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bbbfe0-26a8-4be5-b5c3-376bc102c4c1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:06.980069 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-79bbbfe0-26a8-4be5-b5c3-376bc102c4c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:07.045114 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9836a1c-3fce-4b40-b764-f4c26444d4d8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:07.056119 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad207e25-ccd4-4356-b71b-e453496c72b2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1afda9d6b1d64e6ab400b26124cf1290', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:07.057242 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad207e25-ccd4-4356-b71b-e453496c72b2 service glance] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:07.057807 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad207e25-ccd4-4356-b71b-e453496c72b2 service glance] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:07.058494 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad207e25-ccd4-4356-b71b-e453496c72b2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:07.062357 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ad207e25-ccd4-4356-b71b-e453496c72b2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:07.144834 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bbbfe0-26a8-4be5-b5c3-376bc102c4c1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:07.149579 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eae34e-6156-4b1b-9ba8-82a78bbc1667 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:07.150824 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eae34e-6156-4b1b-9ba8-82a78bbc1667 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:07.151133 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eae34e-6156-4b1b-9ba8-82a78bbc1667 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:07.151463 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eae34e-6156-4b1b-9ba8-82a78bbc1667 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:07.162103 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eae34e-6156-4b1b-9ba8-82a78bbc1667 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:07.163979 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9eae34e-6156-4b1b-9ba8-82a78bbc1667 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:07.177762 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1122/2218] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:16:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:16:07.178960 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eae34e-6156-4b1b-9ba8-82a78bbc1667 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:07.207725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad207e25-ccd4-4356-b71b-e453496c72b2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:07.208846 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad207e25-ccd4-4356-b71b-e453496c72b2 service glance] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:07.231950 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eae34e-6156-4b1b-9ba8-82a78bbc1667 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:07.257521 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffef5dc8-11d4-4969-b02d-73bec9aa1d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:07.259997 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffef5dc8-11d4-4969-b02d-73bec9aa1d5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:07.259997 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffef5dc8-11d4-4969-b02d-73bec9aa1d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:07.259997 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffef5dc8-11d4-4969-b02d-73bec9aa1d5d admin admin] PATH_INFO: `/v3/projects/6782c945d7464b73bf482adaff579710/users/7ea245573911407e92242319231f1c27/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:07.261802 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1097/2219] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:16:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:16:07.265537 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffef5dc8-11d4-4969-b02d-73bec9aa1d5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6782c945d7464b73bf482adaff579710, user_id=7ea245573911407e92242319231f1c27, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:07.268253 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffef5dc8-11d4-4969-b02d-73bec9aa1d5d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:07.311172 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1123/2220] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:16:07 2022] PUT /identity/v3/projects/6782c945d7464b73bf482adaff579710/users/7ea245573911407e92242319231f1c27/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:16:07.323068 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2750c1-504f-445e-8f66-be241f17125c None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:07.323615 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2750c1-504f-445e-8f66-be241f17125c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:07.324031 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2750c1-504f-445e-8f66-be241f17125c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:07.385189 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-dd2750c1-504f-445e-8f66-be241f17125c None None] MFA Rules not processed for user `7ea245573911407e92242319231f1c27`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:16:07.386715 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-dd2750c1-504f-445e-8f66-be241f17125c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:07.501037 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1098/2221] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:16:07 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:16:07.514953 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffef5dc8-11d4-4969-b02d-73bec9aa1d5d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:07.517440 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ffef5dc8-11d4-4969-b02d-73bec9aa1d5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:07.563155 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b01fa-207c-47d5-b93d-c1dd685466c4 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:07.563446 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b01fa-207c-47d5-b93d-c1dd685466c4 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:07.563747 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b01fa-207c-47d5-b93d-c1dd685466c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:07.585006 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:07.618428 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e13b01fa-207c-47d5-b93d-c1dd685466c4 None None] MFA Rules not processed for user `09c23d1c732c452a9597c647ad0241f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:16:07.620661 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e13b01fa-207c-47d5-b93d-c1dd685466c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:07.727255 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffef5dc8-11d4-4969-b02d-73bec9aa1d5d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:07.746922 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6f6f04-70fa-4d78-ad0a-d233823e7e39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:07.748346 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6f6f04-70fa-4d78-ad0a-d233823e7e39 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:07.748640 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6f6f04-70fa-4d78-ad0a-d233823e7e39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:07.749100 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6f6f04-70fa-4d78-ad0a-d233823e7e39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:07.754682 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fd6f6f04-70fa-4d78-ad0a-d233823e7e39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:07.792861 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:07.844014 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1099/2222] 10.222.0.50 () {58 vars in 1009 bytes} [Fri May 27 15:16:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:16:07.949277 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13b01fa-207c-47d5-b93d-c1dd685466c4 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:07.950715 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e13b01fa-207c-47d5-b93d-c1dd685466c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:08.005422 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6f6f04-70fa-4d78-ad0a-d233823e7e39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:08.008431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6f6f04-70fa-4d78-ad0a-d233823e7e39 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:08.034883 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1124/2223] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:16:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:16:08.123644 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13b01fa-207c-47d5-b93d-c1dd685466c4 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:08.131534 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea70fbe-1796-4029-91e2-368110158b96 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18903f2eab7c415096e075d47f0b9a4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:08.132978 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea70fbe-1796-4029-91e2-368110158b96 service swift] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:08.133488 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea70fbe-1796-4029-91e2-368110158b96 service swift] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:08.133774 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea70fbe-1796-4029-91e2-368110158b96 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:08.152411 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea70fbe-1796-4029-91e2-368110158b96 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:08.154682 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea70fbe-1796-4029-91e2-368110158b96 service swift] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:08.169440 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1100/2224] 10.222.0.50 () {60 vars in 1384 bytes} [Fri May 27 15:16:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 15:16:08.374161 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6f6f04-70fa-4d78-ad0a-d233823e7e39 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:08.376144 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fd6f6f04-70fa-4d78-ad0a-d233823e7e39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:08.536794 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6f6f04-70fa-4d78-ad0a-d233823e7e39 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:08.544270 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63687991-a13b-4a66-88b6-bbf3a4bec72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:08.545619 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63687991-a13b-4a66-88b6-bbf3a4bec72b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:08.551179 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63687991-a13b-4a66-88b6-bbf3a4bec72b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:08.551734 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63687991-a13b-4a66-88b6-bbf3a4bec72b admin admin] PATH_INFO: `/v3/projects/ec4c9c703987400ca2228a76cfe52f75` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:08.553480 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63687991-a13b-4a66-88b6-bbf3a4bec72b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec4c9c703987400ca2228a76cfe52f75)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:08.555640 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63687991-a13b-4a66-88b6-bbf3a4bec72b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:08.643681 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-63687991-a13b-4a66-88b6-bbf3a4bec72b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec4c9c703987400ca2228a76cfe52f75', 'request_id': 'req-63687991-a13b-4a66-88b6-bbf3a4bec72b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:08.674735 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-63687991-a13b-4a66-88b6-bbf3a4bec72b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec4c9c703987400ca2228a76cfe52f75', 'request_id': 'req-63687991-a13b-4a66-88b6-bbf3a4bec72b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:08.703346 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-63687991-a13b-4a66-88b6-bbf3a4bec72b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec4c9c703987400ca2228a76cfe52f75', 'request_id': 'req-63687991-a13b-4a66-88b6-bbf3a4bec72b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:08.761874 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-63687991-a13b-4a66-88b6-bbf3a4bec72b admin admin] The token cache is being invalidate because project ec4c9c703987400ca2228a76cfe52f75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:16:08.764779 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-63687991-a13b-4a66-88b6-bbf3a4bec72b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:08.769182 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-63687991-a13b-4a66-88b6-bbf3a4bec72b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:08.779147 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1125/2225] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:16:08 2022] DELETE /identity/v3/projects/ec4c9c703987400ca2228a76cfe52f75 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:16:08.780477 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea70fbe-1796-4029-91e2-368110158b96 service swift] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:08.839643 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea70fbe-1796-4029-91e2-368110158b96 service swift] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:09.071124 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37554125-ad85-4dfc-84c2-ec985f278e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:09.072173 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37554125-ad85-4dfc-84c2-ec985f278e8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:09.072538 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37554125-ad85-4dfc-84c2-ec985f278e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:09.072876 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37554125-ad85-4dfc-84c2-ec985f278e8c admin admin] PATH_INFO: `/v3/users/f6133bf47e974848a2a05f8cee26f52d` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:09.074836 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37554125-ad85-4dfc-84c2-ec985f278e8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6133bf47e974848a2a05f8cee26f52d)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:09.077828 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37554125-ad85-4dfc-84c2-ec985f278e8c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:09.185196 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-37554125-ad85-4dfc-84c2-ec985f278e8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6133bf47e974848a2a05f8cee26f52d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:09.213450 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-37554125-ad85-4dfc-84c2-ec985f278e8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6133bf47e974848a2a05f8cee26f52d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:09.236578 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-37554125-ad85-4dfc-84c2-ec985f278e8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6133bf47e974848a2a05f8cee26f52d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:09.239110 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-37554125-ad85-4dfc-84c2-ec985f278e8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6133bf47e974848a2a05f8cee26f52d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:09.239969 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-37554125-ad85-4dfc-84c2-ec985f278e8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6133bf47e974848a2a05f8cee26f52d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:09.289475 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:09.305562 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1101/2226] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:16:08 2022] DELETE /identity/v3/users/f6133bf47e974848a2a05f8cee26f52d => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:16:09.852727 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63687991-a13b-4a66-88b6-bbf3a4bec72b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:09.858268 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-63687991-a13b-4a66-88b6-bbf3a4bec72b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:09.907062 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63687991-a13b-4a66-88b6-bbf3a4bec72b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:09.938002 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:10.125416 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d30003-c188-47ff-9f22-03587fcb7c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:10.126309 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d30003-c188-47ff-9f22-03587fcb7c2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:10.126595 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d30003-c188-47ff-9f22-03587fcb7c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:10.126890 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d30003-c188-47ff-9f22-03587fcb7c2e admin admin] PATH_INFO: `/v3/projects/7349e8090d1c450d8e7d6fa2f7b9bab3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:10.135835 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d30003-c188-47ff-9f22-03587fcb7c2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7349e8090d1c450d8e7d6fa2f7b9bab3)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:10.139504 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d30003-c188-47ff-9f22-03587fcb7c2e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:10.342285 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37554125-ad85-4dfc-84c2-ec985f278e8c admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:10.344745 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-37554125-ad85-4dfc-84c2-ec985f278e8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:10.347786 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-76d30003-c188-47ff-9f22-03587fcb7c2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7349e8090d1c450d8e7d6fa2f7b9bab3', 'request_id': 'req-76d30003-c188-47ff-9f22-03587fcb7c2e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:10.371815 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-76d30003-c188-47ff-9f22-03587fcb7c2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7349e8090d1c450d8e7d6fa2f7b9bab3', 'request_id': 'req-76d30003-c188-47ff-9f22-03587fcb7c2e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:10.396400 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-76d30003-c188-47ff-9f22-03587fcb7c2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7349e8090d1c450d8e7d6fa2f7b9bab3', 'request_id': 'req-76d30003-c188-47ff-9f22-03587fcb7c2e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:10.436277 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-76d30003-c188-47ff-9f22-03587fcb7c2e admin admin] The token cache is being invalidate because project 7349e8090d1c450d8e7d6fa2f7b9bab3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:16:10.436587 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-76d30003-c188-47ff-9f22-03587fcb7c2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:10.438469 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-76d30003-c188-47ff-9f22-03587fcb7c2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:10.444614 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1126/2227] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:16:09 2022] DELETE /identity/v3/projects/7349e8090d1c450d8e7d6fa2f7b9bab3 => generated 0 bytes in 588 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:16:10.446319 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81082b-33d7-47e9-b357-922c385e6e71 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:10.446715 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81082b-33d7-47e9-b357-922c385e6e71 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:10.447033 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81082b-33d7-47e9-b357-922c385e6e71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:10.487947 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37554125-ad85-4dfc-84c2-ec985f278e8c admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:10.512730 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2f81082b-33d7-47e9-b357-922c385e6e71 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:16:10.513983 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2f81082b-33d7-47e9-b357-922c385e6e71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:10.616794 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07326ddd-dbde-4122-b8ce-61a2f3f71b2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:10.617960 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07326ddd-dbde-4122-b8ce-61a2f3f71b2d service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:10.618320 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07326ddd-dbde-4122-b8ce-61a2f3f71b2d service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:10.619261 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07326ddd-dbde-4122-b8ce-61a2f3f71b2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:10.621888 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-07326ddd-dbde-4122-b8ce-61a2f3f71b2d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:10.711521 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1127/2228] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:16:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:16:10.720057 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2add035-433a-4559-a295-cdb4ea7c63ef None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:10.720600 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2add035-433a-4559-a295-cdb4ea7c63ef None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:10.721380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2add035-433a-4559-a295-cdb4ea7c63ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:10.750643 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a2add035-433a-4559-a295-cdb4ea7c63ef None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:16:10.752048 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a2add035-433a-4559-a295-cdb4ea7c63ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:10.770725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07326ddd-dbde-4122-b8ce-61a2f3f71b2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:10.772228 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07326ddd-dbde-4122-b8ce-61a2f3f71b2d service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:10.779311 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1128/2229] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:16:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:16:10.783189 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2add035-433a-4559-a295-cdb4ea7c63ef None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:10.817496 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2add035-433a-4559-a295-cdb4ea7c63ef None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:10.818653 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1102/2230] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:16:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:16:10.832286 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1605c129-b09b-4c1f-9d18-57b93518af0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:10.833403 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1605c129-b09b-4c1f-9d18-57b93518af0a admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:10.845666 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1605c129-b09b-4c1f-9d18-57b93518af0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:10.845965 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1605c129-b09b-4c1f-9d18-57b93518af0a admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:10.847554 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1605c129-b09b-4c1f-9d18-57b93518af0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:16:10.848773 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1605c129-b09b-4c1f-9d18-57b93518af0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:10.850628 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1605c129-b09b-4c1f-9d18-57b93518af0a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:10.875815 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1129/2231] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:16:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:16:10.880501 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07326ddd-dbde-4122-b8ce-61a2f3f71b2d service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:10.921689 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07326ddd-dbde-4122-b8ce-61a2f3f71b2d service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:10.939009 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d007518-c564-4631-ac5e-9842d5be7240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:10.940064 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d007518-c564-4631-ac5e-9842d5be7240 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:10.941217 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d007518-c564-4631-ac5e-9842d5be7240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:10.941723 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d007518-c564-4631-ac5e-9842d5be7240 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:10.942877 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d007518-c564-4631-ac5e-9842d5be7240 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:10.961088 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d007518-c564-4631-ac5e-9842d5be7240 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:11.044067 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1103/2232] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:16:10 2022] POST /identity/v3/projects => generated 389 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:16:11.048332 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1605c129-b09b-4c1f-9d18-57b93518af0a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:11.073815 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1605c129-b09b-4c1f-9d18-57b93518af0a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:11.123572 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:11.269334 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8630df04-8b16-420d-85c9-f3a62c38be58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:11.270037 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630df04-8b16-420d-85c9-f3a62c38be58 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:11.270322 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630df04-8b16-420d-85c9-f3a62c38be58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:11.270602 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630df04-8b16-420d-85c9-f3a62c38be58 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:11.271345 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8630df04-8b16-420d-85c9-f3a62c38be58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:11.273320 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8630df04-8b16-420d-85c9-f3a62c38be58 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:11.313304 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1130/2233] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:16:11 2022] POST /identity/v3/users => generated 295 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:16:11.317291 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d007518-c564-4631-ac5e-9842d5be7240 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:11.336262 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d007518-c564-4631-ac5e-9842d5be7240 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:11.360498 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d59252-9a8e-4838-87f3-a13febdc31f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:11.365838 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d59252-9a8e-4838-87f3-a13febdc31f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:11.366166 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d59252-9a8e-4838-87f3-a13febdc31f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:11.366454 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d59252-9a8e-4838-87f3-a13febdc31f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:11.367291 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d59252-9a8e-4838-87f3-a13febdc31f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:11.369102 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d59252-9a8e-4838-87f3-a13febdc31f8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:11.420237 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1104/2234] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:16:11 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:16:11.421678 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8630df04-8b16-420d-85c9-f3a62c38be58 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:11.445600 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8630df04-8b16-420d-85c9-f3a62c38be58 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:11.455014 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e87f58-f862-445f-936d-b3592e94a06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:11.455759 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e87f58-f862-445f-936d-b3592e94a06e admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:11.456044 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e87f58-f862-445f-936d-b3592e94a06e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:11.456346 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e87f58-f862-445f-936d-b3592e94a06e admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:11.457096 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e87f58-f862-445f-936d-b3592e94a06e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:11.470443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e87f58-f862-445f-936d-b3592e94a06e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:11.492417 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1131/2235] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:16:11 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:16:11.497121 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d59252-9a8e-4838-87f3-a13febdc31f8 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:11.510841 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d59252-9a8e-4838-87f3-a13febdc31f8 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:11.523378 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05c060-8e42-47b2-b67d-6f3e61a6951b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:11.526656 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05c060-8e42-47b2-b67d-6f3e61a6951b admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:11.526995 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05c060-8e42-47b2-b67d-6f3e61a6951b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:11.527289 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05c060-8e42-47b2-b67d-6f3e61a6951b admin admin] PATH_INFO: `/v3/projects/88cd584cbe1b4eafa18c6a0b2c6ede71/users/7554c151737b4c4fa2ed3391a83f3803/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:11.566562 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a05c060-8e42-47b2-b67d-6f3e61a6951b admin admin] RBAC: Authorizing `identity:create_grant(project_id=88cd584cbe1b4eafa18c6a0b2c6ede71, user_id=7554c151737b4c4fa2ed3391a83f3803, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:11.570175 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a05c060-8e42-47b2-b67d-6f3e61a6951b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:11.641968 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1105/2236] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:16:11 2022] PUT /identity/v3/projects/88cd584cbe1b4eafa18c6a0b2c6ede71/users/7554c151737b4c4fa2ed3391a83f3803/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:16:11.656158 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b11ae-0878-4b65-a0cd-75f6a2dcc589 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:11.656774 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b11ae-0878-4b65-a0cd-75f6a2dcc589 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:11.657235 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b11ae-0878-4b65-a0cd-75f6a2dcc589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:11.747493 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7a6b11ae-0878-4b65-a0cd-75f6a2dcc589 None None] MFA Rules not processed for user `7554c151737b4c4fa2ed3391a83f3803`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:16:11.749865 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7a6b11ae-0878-4b65-a0cd-75f6a2dcc589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:11.954832 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1132/2237] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:16:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:16:11.959487 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05c060-8e42-47b2-b67d-6f3e61a6951b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:11.988961 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:12.178172 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6b11ae-0878-4b65-a0cd-75f6a2dcc589 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:12.180072 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7a6b11ae-0878-4b65-a0cd-75f6a2dcc589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:12.191464 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05c060-8e42-47b2-b67d-6f3e61a6951b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:12.207032 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a85b75-5dbc-4972-9c11-1cfca00c0961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:12.210017 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a85b75-5dbc-4972-9c11-1cfca00c0961 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:12.210359 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a85b75-5dbc-4972-9c11-1cfca00c0961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:12.210638 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a85b75-5dbc-4972-9c11-1cfca00c0961 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:12.220939 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a85b75-5dbc-4972-9c11-1cfca00c0961 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:12.222926 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a85b75-5dbc-4972-9c11-1cfca00c0961 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:12.274324 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1106/2238] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:16:11 2022] POST /identity/v3/projects => generated 393 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:16:12.276303 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a85b75-5dbc-4972-9c11-1cfca00c0961 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:12.315060 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6b11ae-0878-4b65-a0cd-75f6a2dcc589 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:12.434999 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:12.438262 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a85b75-5dbc-4972-9c11-1cfca00c0961 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:12.472003 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7365986b-221f-4f6c-9cfe-d730dabccf41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:12.472818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365986b-221f-4f6c-9cfe-d730dabccf41 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:12.474616 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365986b-221f-4f6c-9cfe-d730dabccf41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:12.475152 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365986b-221f-4f6c-9cfe-d730dabccf41 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:12.476730 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7365986b-221f-4f6c-9cfe-d730dabccf41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:12.477924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc566c1-303a-4471-829b-68d259231e55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:12.479815 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc566c1-303a-4471-829b-68d259231e55 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:12.482676 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7365986b-221f-4f6c-9cfe-d730dabccf41 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:12.484734 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc566c1-303a-4471-829b-68d259231e55 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:12.485153 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc566c1-303a-4471-829b-68d259231e55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:12.491914 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7fc566c1-303a-4471-829b-68d259231e55 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:12.545193 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1107/2239] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:16:12 2022] POST /identity/v3/users => generated 303 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:16:12.549358 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7365986b-221f-4f6c-9cfe-d730dabccf41 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:12.569539 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7365986b-221f-4f6c-9cfe-d730dabccf41 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:12.582816 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6afd7e-33d5-4b26-a533-c22cbc389ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:12.591758 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6afd7e-33d5-4b26-a533-c22cbc389ab5 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:12.593960 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6afd7e-33d5-4b26-a533-c22cbc389ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:12.596905 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6afd7e-33d5-4b26-a533-c22cbc389ab5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:12.599303 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6afd7e-33d5-4b26-a533-c22cbc389ab5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:12.602586 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6afd7e-33d5-4b26-a533-c22cbc389ab5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:12.627134 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1108/2240] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:16:12 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:16:12.630894 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6afd7e-33d5-4b26-a533-c22cbc389ab5 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:12.669458 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6afd7e-33d5-4b26-a533-c22cbc389ab5 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:12.680820 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0512e42-e98d-468f-9825-b21177cc1d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:12.681934 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0512e42-e98d-468f-9825-b21177cc1d17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:12.682415 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0512e42-e98d-468f-9825-b21177cc1d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:12.682883 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0512e42-e98d-468f-9825-b21177cc1d17 admin admin] PATH_INFO: `/v3/projects/522ec32464a14cb19b8fe3bc676827ae/users/4f3329fa705e47178a6d3513eb0251d8/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:12.685916 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc566c1-303a-4471-829b-68d259231e55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:12.688231 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc566c1-303a-4471-829b-68d259231e55 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:12.714117 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1133/2241] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:16:12 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:16:12.718767 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0512e42-e98d-468f-9825-b21177cc1d17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=522ec32464a14cb19b8fe3bc676827ae, user_id=4f3329fa705e47178a6d3513eb0251d8, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:12.721540 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0512e42-e98d-468f-9825-b21177cc1d17 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:12.742084 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1109/2242] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:16:12 2022] PUT /identity/v3/projects/522ec32464a14cb19b8fe3bc676827ae/users/4f3329fa705e47178a6d3513eb0251d8/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:16:12.743493 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc566c1-303a-4471-829b-68d259231e55 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:12.777566 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc566c1-303a-4471-829b-68d259231e55 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:12.904722 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3af7d0-0cd3-47aa-80d8-5e1c47c47dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:12.905441 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3af7d0-0cd3-47aa-80d8-5e1c47c47dd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:12.905771 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3af7d0-0cd3-47aa-80d8-5e1c47c47dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:12.906107 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3af7d0-0cd3-47aa-80d8-5e1c47c47dd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:12.906836 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3af7d0-0cd3-47aa-80d8-5e1c47c47dd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:12.909023 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3af7d0-0cd3-47aa-80d8-5e1c47c47dd0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:12.931981 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1134/2243] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:16:12 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:16:12.933950 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0512e42-e98d-468f-9825-b21177cc1d17 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:12.951267 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0512e42-e98d-468f-9825-b21177cc1d17 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:12.957224 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed4cb1-8310-4940-88d8-3a7e63cba8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:12.960795 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed4cb1-8310-4940-88d8-3a7e63cba8a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:12.961090 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed4cb1-8310-4940-88d8-3a7e63cba8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:12.961512 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed4cb1-8310-4940-88d8-3a7e63cba8a5 admin admin] PATH_INFO: `/v3/projects/522ec32464a14cb19b8fe3bc676827ae/users/4f3329fa705e47178a6d3513eb0251d8/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:12.964229 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ed4cb1-8310-4940-88d8-3a7e63cba8a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=522ec32464a14cb19b8fe3bc676827ae, user_id=4f3329fa705e47178a6d3513eb0251d8, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:12.967155 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ed4cb1-8310-4940-88d8-3a7e63cba8a5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:12.988007 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1110/2244] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:16:12 2022] PUT /identity/v3/projects/522ec32464a14cb19b8fe3bc676827ae/users/4f3329fa705e47178a6d3513eb0251d8/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:16:12.998485 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b3d43-366e-4239-9f6f-6880aac4c1b9 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:12.999040 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b3d43-366e-4239-9f6f-6880aac4c1b9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:12.999141 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b3d43-366e-4239-9f6f-6880aac4c1b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:13.030744 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-727b3d43-366e-4239-9f6f-6880aac4c1b9 None None] MFA Rules not processed for user `4f3329fa705e47178a6d3513eb0251d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:16:13.032728 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-727b3d43-366e-4239-9f6f-6880aac4c1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:13.064292 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed4cb1-8310-4940-88d8-3a7e63cba8a5 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:13.066140 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-03ed4cb1-8310-4940-88d8-3a7e63cba8a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:13.165032 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:13.206346 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1135/2245] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:16:12 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:16:13.210939 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed4cb1-8310-4940-88d8-3a7e63cba8a5 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:13.216254 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b3d43-366e-4239-9f6f-6880aac4c1b9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:13.218673 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f750c05-54f2-4761-8992-1d65ebd4dd24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:13.219759 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f750c05-54f2-4761-8992-1d65ebd4dd24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:13.220007 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-727b3d43-366e-4239-9f6f-6880aac4c1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:13.220111 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f750c05-54f2-4761-8992-1d65ebd4dd24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:13.220296 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f750c05-54f2-4761-8992-1d65ebd4dd24 admin admin] PATH_INFO: `/v3/users/fd592734e7124bb8b1fdc7b351ae018b` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:13.221639 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f750c05-54f2-4761-8992-1d65ebd4dd24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd592734e7124bb8b1fdc7b351ae018b)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:13.223820 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f750c05-54f2-4761-8992-1d65ebd4dd24 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:13.251761 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:13.321422 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4f750c05-54f2-4761-8992-1d65ebd4dd24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd592734e7124bb8b1fdc7b351ae018b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:13.356889 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b3d43-366e-4239-9f6f-6880aac4c1b9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:13.362611 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4f750c05-54f2-4761-8992-1d65ebd4dd24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd592734e7124bb8b1fdc7b351ae018b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:13.396528 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0fab79-5242-4412-958b-215d136afb31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:13.397247 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0fab79-5242-4412-958b-215d136afb31 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:13.397543 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0fab79-5242-4412-958b-215d136afb31 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:13.397867 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0fab79-5242-4412-958b-215d136afb31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:13.398382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4f750c05-54f2-4761-8992-1d65ebd4dd24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd592734e7124bb8b1fdc7b351ae018b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:13.400713 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4f750c05-54f2-4761-8992-1d65ebd4dd24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd592734e7124bb8b1fdc7b351ae018b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:13.401357 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4f750c05-54f2-4761-8992-1d65ebd4dd24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd592734e7124bb8b1fdc7b351ae018b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:13.422148 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0fab79-5242-4412-958b-215d136afb31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:13.423667 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0fab79-5242-4412-958b-215d136afb31 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:13.443153 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1136/2246] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:16:13 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:16:13.451344 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1111/2247] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:16:13 2022] DELETE /identity/v3/users/fd592734e7124bb8b1fdc7b351ae018b => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:16:13.456755 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2aa7e-802f-4e1b-8e64-32536415b94f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:13.458575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2aa7e-802f-4e1b-8e64-32536415b94f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:13.459076 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2aa7e-802f-4e1b-8e64-32536415b94f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:13.500824 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-95b2aa7e-802f-4e1b-8e64-32536415b94f None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:16:13.502053 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-95b2aa7e-802f-4e1b-8e64-32536415b94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:13.520095 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f750c05-54f2-4761-8992-1d65ebd4dd24 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:13.522235 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4f750c05-54f2-4761-8992-1d65ebd4dd24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:13.600051 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1137/2248] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:16:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:16:13.613292 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b2aa7e-802f-4e1b-8e64-32536415b94f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:13.616653 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-95b2aa7e-802f-4e1b-8e64-32536415b94f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:13.625019 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f750c05-54f2-4761-8992-1d65ebd4dd24 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:13.634823 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7abedc6-f281-40cf-a6c4-b4e5aca92dab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:13.635538 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7abedc6-f281-40cf-a6c4-b4e5aca92dab service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:13.635817 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7abedc6-f281-40cf-a6c4-b4e5aca92dab service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:13.636365 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7abedc6-f281-40cf-a6c4-b4e5aca92dab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:13.639408 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e7abedc6-f281-40cf-a6c4-b4e5aca92dab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:13.720803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b2aa7e-802f-4e1b-8e64-32536415b94f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:13.724666 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f6f1b-fb5b-4adb-b505-bcd2c9641f98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:13.725867 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f6f1b-fb5b-4adb-b505-bcd2c9641f98 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:13.726215 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f6f1b-fb5b-4adb-b505-bcd2c9641f98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:13.726566 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f6f1b-fb5b-4adb-b505-bcd2c9641f98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:13.738463 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2f6f1b-fb5b-4adb-b505-bcd2c9641f98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:13.740380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2f6f1b-fb5b-4adb-b505-bcd2c9641f98 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:13.754060 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1138/2249] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:16:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:16:13.760046 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7abedc6-f281-40cf-a6c4-b4e5aca92dab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:13.761791 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7abedc6-f281-40cf-a6c4-b4e5aca92dab service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:13.789093 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1112/2250] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:16:13 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:16:14.374196 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f6f1b-fb5b-4adb-b505-bcd2c9641f98 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:14.375791 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2c2f6f1b-fb5b-4adb-b505-bcd2c9641f98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:14.401359 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f6f1b-fb5b-4adb-b505-bcd2c9641f98 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:14.417809 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f0ca61-e092-4618-a9f3-443afb0853f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:14.418771 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f0ca61-e092-4618-a9f3-443afb0853f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:14.432963 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f0ca61-e092-4618-a9f3-443afb0853f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:14.436262 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f0ca61-e092-4618-a9f3-443afb0853f8 admin admin] PATH_INFO: `/v3/projects/5de7b2affd3f432081e89391e3adce99` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:14.444430 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f0ca61-e092-4618-a9f3-443afb0853f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5de7b2affd3f432081e89391e3adce99)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:14.456188 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f0ca61-e092-4618-a9f3-443afb0853f8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:14.549277 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e1f0ca61-e092-4618-a9f3-443afb0853f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5de7b2affd3f432081e89391e3adce99', 'request_id': 'req-e1f0ca61-e092-4618-a9f3-443afb0853f8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:14.555417 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:14.575564 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e1f0ca61-e092-4618-a9f3-443afb0853f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5de7b2affd3f432081e89391e3adce99', 'request_id': 'req-e1f0ca61-e092-4618-a9f3-443afb0853f8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:14.591190 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e1f0ca61-e092-4618-a9f3-443afb0853f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5de7b2affd3f432081e89391e3adce99', 'request_id': 'req-e1f0ca61-e092-4618-a9f3-443afb0853f8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:14.611526 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e1f0ca61-e092-4618-a9f3-443afb0853f8 admin admin] The token cache is being invalidate because project 5de7b2affd3f432081e89391e3adce99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:16:14.611526 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e1f0ca61-e092-4618-a9f3-443afb0853f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:14.611526 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e1f0ca61-e092-4618-a9f3-443afb0853f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:14.613702 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1139/2251] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:16:14 2022] DELETE /identity/v3/projects/5de7b2affd3f432081e89391e3adce99 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:16:14.618738 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931fc0e9-0cf9-468d-a06a-dc2990fc404d None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:14.619340 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931fc0e9-0cf9-468d-a06a-dc2990fc404d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:14.619536 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931fc0e9-0cf9-468d-a06a-dc2990fc404d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:14.644541 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-931fc0e9-0cf9-468d-a06a-dc2990fc404d None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:16:14.645865 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-931fc0e9-0cf9-468d-a06a-dc2990fc404d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:14.666564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:14.748439 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1113/2252] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:16:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:16:14.756035 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39632f-82e9-43eb-9aa4-0d70990ff7eb None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:14.756690 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39632f-82e9-43eb-9aa4-0d70990ff7eb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:14.757357 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c39632f-82e9-43eb-9aa4-0d70990ff7eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:14.797413 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0c39632f-82e9-43eb-9aa4-0d70990ff7eb None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:16:14.801501 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0c39632f-82e9-43eb-9aa4-0d70990ff7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:14.820950 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1140/2253] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:16:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:16:14.822926 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931fc0e9-0cf9-468d-a06a-dc2990fc404d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:14.833452 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931fc0e9-0cf9-468d-a06a-dc2990fc404d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:14.851504 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ae7e68-ae5f-4b46-97ce-7f843c0d87e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:14.852193 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae7e68-ae5f-4b46-97ce-7f843c0d87e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:14.852529 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae7e68-ae5f-4b46-97ce-7f843c0d87e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:14.852846 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae7e68-ae5f-4b46-97ce-7f843c0d87e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:14.853589 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ae7e68-ae5f-4b46-97ce-7f843c0d87e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:16:14.853926 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ae7e68-ae5f-4b46-97ce-7f843c0d87e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:14.855606 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ae7e68-ae5f-4b46-97ce-7f843c0d87e0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:14.887080 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c39632f-82e9-43eb-9aa4-0d70990ff7eb None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:14.891232 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1114/2254] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:16:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:16:14.904191 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c39632f-82e9-43eb-9aa4-0d70990ff7eb None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:14.915911 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ea6602-9f91-42c2-a1b4-ad51725fff22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:14.916729 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea6602-9f91-42c2-a1b4-ad51725fff22 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:14.917145 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea6602-9f91-42c2-a1b4-ad51725fff22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:14.917702 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea6602-9f91-42c2-a1b4-ad51725fff22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:14.918522 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ea6602-9f91-42c2-a1b4-ad51725fff22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:14.920791 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ea6602-9f91-42c2-a1b4-ad51725fff22 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:14.979846 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1141/2255] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:16:14 2022] POST /identity/v3/projects => generated 379 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:16:14.982970 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ae7e68-ae5f-4b46-97ce-7f843c0d87e0 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:14.998474 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ae7e68-ae5f-4b46-97ce-7f843c0d87e0 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:15.094828 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d2ccd5-ca12-45ca-b76b-0de28ea13577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:15.095796 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d2ccd5-ca12-45ca-b76b-0de28ea13577 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:15.096105 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d2ccd5-ca12-45ca-b76b-0de28ea13577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:15.096399 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d2ccd5-ca12-45ca-b76b-0de28ea13577 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:15.097120 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d2ccd5-ca12-45ca-b76b-0de28ea13577 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:15.099179 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d2ccd5-ca12-45ca-b76b-0de28ea13577 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:15.135142 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1115/2256] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:16:14 2022] POST /identity/v3/users => generated 290 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:16:15.137531 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ea6602-9f91-42c2-a1b4-ad51725fff22 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:15.164397 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ea6602-9f91-42c2-a1b4-ad51725fff22 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:15.170011 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667838d4-b041-4da2-aeb9-4eb8049df5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:15.170464 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667838d4-b041-4da2-aeb9-4eb8049df5f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:15.170751 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667838d4-b041-4da2-aeb9-4eb8049df5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:15.171090 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667838d4-b041-4da2-aeb9-4eb8049df5f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:15.171921 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667838d4-b041-4da2-aeb9-4eb8049df5f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:15.173623 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667838d4-b041-4da2-aeb9-4eb8049df5f7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:15.190433 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1142/2257] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:16:15 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:16:15.191852 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d2ccd5-ca12-45ca-b76b-0de28ea13577 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:15.203478 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d2ccd5-ca12-45ca-b76b-0de28ea13577 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:15.209286 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52328903-bc2a-4fcf-a024-0a08b6ec489e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:15.209581 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52328903-bc2a-4fcf-a024-0a08b6ec489e admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:15.209872 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52328903-bc2a-4fcf-a024-0a08b6ec489e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:15.210173 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52328903-bc2a-4fcf-a024-0a08b6ec489e admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:15.211237 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52328903-bc2a-4fcf-a024-0a08b6ec489e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:15.213263 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52328903-bc2a-4fcf-a024-0a08b6ec489e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:15.227450 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1116/2258] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:16:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:16:15.231935 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667838d4-b041-4da2-aeb9-4eb8049df5f7 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:15.242952 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667838d4-b041-4da2-aeb9-4eb8049df5f7 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:15.250135 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb905cf7-d771-4d1e-b8c3-47a1b6c6434b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:15.250135 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb905cf7-d771-4d1e-b8c3-47a1b6c6434b admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:15.250135 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb905cf7-d771-4d1e-b8c3-47a1b6c6434b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:15.250739 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb905cf7-d771-4d1e-b8c3-47a1b6c6434b admin admin] PATH_INFO: `/v3/projects/f88bb4c8b75c4ad6be62f2cf5cf509c4/users/af74115f423c43fdb2544876be4e5622/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:15.271622 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb905cf7-d771-4d1e-b8c3-47a1b6c6434b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f88bb4c8b75c4ad6be62f2cf5cf509c4, user_id=af74115f423c43fdb2544876be4e5622, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:15.275952 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb905cf7-d771-4d1e-b8c3-47a1b6c6434b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:15.292946 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1143/2259] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:16:15 2022] PUT /identity/v3/projects/f88bb4c8b75c4ad6be62f2cf5cf509c4/users/af74115f423c43fdb2544876be4e5622/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:16:15.306285 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ab8dc-1d2c-4550-bdc2-ebe269c713f8 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:15.306285 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ab8dc-1d2c-4550-bdc2-ebe269c713f8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:15.306285 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ab8dc-1d2c-4550-bdc2-ebe269c713f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:15.331788 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ae3ab8dc-1d2c-4550-bdc2-ebe269c713f8 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:16:15.333029 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ae3ab8dc-1d2c-4550-bdc2-ebe269c713f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:15.405926 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1117/2260] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:16:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:16:15.418256 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb905cf7-d771-4d1e-b8c3-47a1b6c6434b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:15.420045 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bb905cf7-d771-4d1e-b8c3-47a1b6c6434b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:15.513270 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb905cf7-d771-4d1e-b8c3-47a1b6c6434b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:15.517992 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f3279f-003e-40a7-83da-3a542ca65067 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:15.518758 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3279f-003e-40a7-83da-3a542ca65067 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:15.519073 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3279f-003e-40a7-83da-3a542ca65067 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:15.519353 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f3279f-003e-40a7-83da-3a542ca65067 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:15.530839 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f3279f-003e-40a7-83da-3a542ca65067 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:15.532321 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f3279f-003e-40a7-83da-3a542ca65067 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:15.546885 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1144/2261] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:16:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:16:17.754436 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ab8dc-1d2c-4550-bdc2-ebe269c713f8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:17.755909 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ae3ab8dc-1d2c-4550-bdc2-ebe269c713f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:17.769081 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ab8dc-1d2c-4550-bdc2-ebe269c713f8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:17.779450 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1dbd0a-1928-4e21-b26c-d3d89ae14eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:17.780345 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1dbd0a-1928-4e21-b26c-d3d89ae14eb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:17.780634 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1dbd0a-1928-4e21-b26c-d3d89ae14eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:17.780912 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1dbd0a-1928-4e21-b26c-d3d89ae14eb5 admin admin] PATH_INFO: `/v3/users/081d618f848e4a119df5954927a6371e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:17.782138 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1dbd0a-1928-4e21-b26c-d3d89ae14eb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=081d618f848e4a119df5954927a6371e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:17.784181 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1dbd0a-1928-4e21-b26c-d3d89ae14eb5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:17.804291 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:17.855077 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4e1dbd0a-1928-4e21-b26c-d3d89ae14eb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '081d618f848e4a119df5954927a6371e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:17.866923 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4e1dbd0a-1928-4e21-b26c-d3d89ae14eb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '081d618f848e4a119df5954927a6371e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:17.887475 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4e1dbd0a-1928-4e21-b26c-d3d89ae14eb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '081d618f848e4a119df5954927a6371e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:17.888253 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4e1dbd0a-1928-4e21-b26c-d3d89ae14eb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '081d618f848e4a119df5954927a6371e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:17.888884 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4e1dbd0a-1928-4e21-b26c-d3d89ae14eb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '081d618f848e4a119df5954927a6371e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:17.916177 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1118/2262] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:16:17 2022] DELETE /identity/v3/users/081d618f848e4a119df5954927a6371e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:16:17.920707 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87de128-ce2e-461b-9fea-4c4fa82d1715 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:17.921568 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87de128-ce2e-461b-9fea-4c4fa82d1715 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:17.922034 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87de128-ce2e-461b-9fea-4c4fa82d1715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:17.958948 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a87de128-ce2e-461b-9fea-4c4fa82d1715 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:16:17.960064 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a87de128-ce2e-461b-9fea-4c4fa82d1715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:18.110359 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24677f-0568-43e6-9932-8092f7504d2f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:18.110644 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24677f-0568-43e6-9932-8092f7504d2f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:18.110956 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d24677f-0568-43e6-9932-8092f7504d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:18.145672 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-6d24677f-0568-43e6-9932-8092f7504d2f None None] MFA Rules not processed for user `34ac2a8ad2e343838f8e3e3285ec5f55`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:16:18.151927 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6d24677f-0568-43e6-9932-8092f7504d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:18.167883 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1145/2263] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:16:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:16:18.177754 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87de128-ce2e-461b-9fea-4c4fa82d1715 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:18.184800 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a87de128-ce2e-461b-9fea-4c4fa82d1715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:18.260459 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:18.286642 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1119/2264] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:16:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:16:18.314092 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87de128-ce2e-461b-9fea-4c4fa82d1715 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:18.319044 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519f1905-ee03-4587-896d-1adefcbc0ac2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:18.320105 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f1905-ee03-4587-896d-1adefcbc0ac2 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:18.320609 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f1905-ee03-4587-896d-1adefcbc0ac2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:18.321090 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519f1905-ee03-4587-896d-1adefcbc0ac2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:18.334723 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f1905-ee03-4587-896d-1adefcbc0ac2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:18.336703 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519f1905-ee03-4587-896d-1adefcbc0ac2 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:18.364325 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1146/2265] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:16:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:16:18.685951 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d24677f-0568-43e6-9932-8092f7504d2f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:18.687666 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6d24677f-0568-43e6-9932-8092f7504d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:18.707804 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d24677f-0568-43e6-9932-8092f7504d2f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:18.715142 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf07db-7cba-4e55-8e11-5cdec665a3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:18.716144 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf07db-7cba-4e55-8e11-5cdec665a3f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:18.716495 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf07db-7cba-4e55-8e11-5cdec665a3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:18.716827 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abf07db-7cba-4e55-8e11-5cdec665a3f9 admin admin] PATH_INFO: `/v3/users/7554c151737b4c4fa2ed3391a83f3803` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:18.719632 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf07db-7cba-4e55-8e11-5cdec665a3f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7554c151737b4c4fa2ed3391a83f3803)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:18.725478 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abf07db-7cba-4e55-8e11-5cdec665a3f9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:18.835549 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4abf07db-7cba-4e55-8e11-5cdec665a3f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7554c151737b4c4fa2ed3391a83f3803'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:18.851921 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4abf07db-7cba-4e55-8e11-5cdec665a3f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7554c151737b4c4fa2ed3391a83f3803'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:18.879361 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4abf07db-7cba-4e55-8e11-5cdec665a3f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7554c151737b4c4fa2ed3391a83f3803'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:18.881620 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4abf07db-7cba-4e55-8e11-5cdec665a3f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7554c151737b4c4fa2ed3391a83f3803'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:18.883410 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4abf07db-7cba-4e55-8e11-5cdec665a3f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7554c151737b4c4fa2ed3391a83f3803'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:18.953748 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1120/2266] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:16:18 2022] DELETE /identity/v3/users/7554c151737b4c4fa2ed3391a83f3803 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:16:18.958710 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb17b3f-7866-4d49-a98e-ecadc9ad29c7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:18.959656 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb17b3f-7866-4d49-a98e-ecadc9ad29c7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:18.960257 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb17b3f-7866-4d49-a98e-ecadc9ad29c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:19.004426 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4eb17b3f-7866-4d49-a98e-ecadc9ad29c7 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:16:19.021746 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4eb17b3f-7866-4d49-a98e-ecadc9ad29c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:19.189144 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1147/2267] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:16:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:16:19.207741 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf07db-7cba-4e55-8e11-5cdec665a3f9 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:19.209199 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4abf07db-7cba-4e55-8e11-5cdec665a3f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:19.241735 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:19.338439 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abf07db-7cba-4e55-8e11-5cdec665a3f9 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:19.342260 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed9cf4-5ff7-4390-9119-6e9224bcfd6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:19.343338 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed9cf4-5ff7-4390-9119-6e9224bcfd6e service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:19.343678 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed9cf4-5ff7-4390-9119-6e9224bcfd6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:19.344053 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed9cf4-5ff7-4390-9119-6e9224bcfd6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:19.365355 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ed9cf4-5ff7-4390-9119-6e9224bcfd6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:19.366823 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ed9cf4-5ff7-4390-9119-6e9224bcfd6e service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:19.394640 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1121/2268] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:16:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:16:19.829779 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb17b3f-7866-4d49-a98e-ecadc9ad29c7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:19.831288 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4eb17b3f-7866-4d49-a98e-ecadc9ad29c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:19.856615 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb17b3f-7866-4d49-a98e-ecadc9ad29c7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:19.861445 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88dea2d-d3a6-4696-9051-e80c56e0a082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:19.862333 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88dea2d-d3a6-4696-9051-e80c56e0a082 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:19.862624 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88dea2d-d3a6-4696-9051-e80c56e0a082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:19.862933 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88dea2d-d3a6-4696-9051-e80c56e0a082 admin admin] PATH_INFO: `/v3/projects/cb7a685d4f744bb98d1b3063d03db0e7` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:19.867813 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88dea2d-d3a6-4696-9051-e80c56e0a082 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb7a685d4f744bb98d1b3063d03db0e7)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:19.870367 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88dea2d-d3a6-4696-9051-e80c56e0a082 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:20.017945 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d88dea2d-d3a6-4696-9051-e80c56e0a082 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb7a685d4f744bb98d1b3063d03db0e7', 'request_id': 'req-d88dea2d-d3a6-4696-9051-e80c56e0a082'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:20.034140 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d88dea2d-d3a6-4696-9051-e80c56e0a082 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb7a685d4f744bb98d1b3063d03db0e7', 'request_id': 'req-d88dea2d-d3a6-4696-9051-e80c56e0a082'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:20.048921 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d88dea2d-d3a6-4696-9051-e80c56e0a082 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb7a685d4f744bb98d1b3063d03db0e7', 'request_id': 'req-d88dea2d-d3a6-4696-9051-e80c56e0a082'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:20.088731 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d88dea2d-d3a6-4696-9051-e80c56e0a082 admin admin] The token cache is being invalidate because project cb7a685d4f744bb98d1b3063d03db0e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:16:20.088731 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d88dea2d-d3a6-4696-9051-e80c56e0a082 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:20.089452 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d88dea2d-d3a6-4696-9051-e80c56e0a082 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:20.096400 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1148/2269] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:16:19 2022] DELETE /identity/v3/projects/cb7a685d4f744bb98d1b3063d03db0e7 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:16:20.104442 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed9cf4-5ff7-4390-9119-6e9224bcfd6e service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:20.131655 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed9cf4-5ff7-4390-9119-6e9224bcfd6e service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:20.272980 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88dea2d-d3a6-4696-9051-e80c56e0a082 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:20.275006 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d88dea2d-d3a6-4696-9051-e80c56e0a082 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:20.373836 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3db34a-461d-4bba-a029-e351453dab5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:20.374737 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3db34a-461d-4bba-a029-e351453dab5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:20.375055 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3db34a-461d-4bba-a029-e351453dab5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:20.375846 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3db34a-461d-4bba-a029-e351453dab5b admin admin] PATH_INFO: `/v3/users/9f4f2d95083544a78dc599ed6849e05e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:20.379533 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3db34a-461d-4bba-a029-e351453dab5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f4f2d95083544a78dc599ed6849e05e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:20.382797 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3db34a-461d-4bba-a029-e351453dab5b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:20.534633 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:20.542265 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ee3db34a-461d-4bba-a029-e351453dab5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f4f2d95083544a78dc599ed6849e05e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:20.558117 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ee3db34a-461d-4bba-a029-e351453dab5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f4f2d95083544a78dc599ed6849e05e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:20.588666 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ee3db34a-461d-4bba-a029-e351453dab5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f4f2d95083544a78dc599ed6849e05e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:20.596704 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ee3db34a-461d-4bba-a029-e351453dab5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f4f2d95083544a78dc599ed6849e05e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:20.598553 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ee3db34a-461d-4bba-a029-e351453dab5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f4f2d95083544a78dc599ed6849e05e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:20.608677 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88dea2d-d3a6-4696-9051-e80c56e0a082 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:20.630325 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaa6e28-8b92-4ba6-92a7-eb4b405700e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:20.632996 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa6e28-8b92-4ba6-92a7-eb4b405700e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:20.633299 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa6e28-8b92-4ba6-92a7-eb4b405700e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:20.633588 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa6e28-8b92-4ba6-92a7-eb4b405700e9 admin admin] PATH_INFO: `/v3/projects/88cd584cbe1b4eafa18c6a0b2c6ede71` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:20.644062 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaa6e28-8b92-4ba6-92a7-eb4b405700e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88cd584cbe1b4eafa18c6a0b2c6ede71)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:20.645874 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1122/2270] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:16:20 2022] DELETE /identity/v3/users/9f4f2d95083544a78dc599ed6849e05e => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:16:20.648925 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaa6e28-8b92-4ba6-92a7-eb4b405700e9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:20.823163 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2eaa6e28-8b92-4ba6-92a7-eb4b405700e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88cd584cbe1b4eafa18c6a0b2c6ede71', 'request_id': 'req-2eaa6e28-8b92-4ba6-92a7-eb4b405700e9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:20.833325 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2eaa6e28-8b92-4ba6-92a7-eb4b405700e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88cd584cbe1b4eafa18c6a0b2c6ede71', 'request_id': 'req-2eaa6e28-8b92-4ba6-92a7-eb4b405700e9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:20.844406 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2eaa6e28-8b92-4ba6-92a7-eb4b405700e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88cd584cbe1b4eafa18c6a0b2c6ede71', 'request_id': 'req-2eaa6e28-8b92-4ba6-92a7-eb4b405700e9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:20.930900 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2eaa6e28-8b92-4ba6-92a7-eb4b405700e9 admin admin] The token cache is being invalidate because project 88cd584cbe1b4eafa18c6a0b2c6ede71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:16:20.934285 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2eaa6e28-8b92-4ba6-92a7-eb4b405700e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:20.935951 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2eaa6e28-8b92-4ba6-92a7-eb4b405700e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:20.939851 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1149/2271] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:16:20 2022] DELETE /identity/v3/projects/88cd584cbe1b4eafa18c6a0b2c6ede71 => generated 0 bytes in 667 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:16:20.944431 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3db34a-461d-4bba-a029-e351453dab5b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:20.947229 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ee3db34a-461d-4bba-a029-e351453dab5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:21.058169 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:21.202485 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3db34a-461d-4bba-a029-e351453dab5b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:21.432729 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448bbda1-9c94-4a4a-b0a0-723176f45e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:21.438357 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448bbda1-9c94-4a4a-b0a0-723176f45e82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:21.438642 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448bbda1-9c94-4a4a-b0a0-723176f45e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:21.438945 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448bbda1-9c94-4a4a-b0a0-723176f45e82 admin admin] PATH_INFO: `/v3/users/4f3329fa705e47178a6d3513eb0251d8` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:21.441041 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448bbda1-9c94-4a4a-b0a0-723176f45e82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f3329fa705e47178a6d3513eb0251d8)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:21.443017 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448bbda1-9c94-4a4a-b0a0-723176f45e82 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:21.504005 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaa6e28-8b92-4ba6-92a7-eb4b405700e9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:21.505779 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2eaa6e28-8b92-4ba6-92a7-eb4b405700e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:21.539744 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaa6e28-8b92-4ba6-92a7-eb4b405700e9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:21.548590 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-448bbda1-9c94-4a4a-b0a0-723176f45e82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f3329fa705e47178a6d3513eb0251d8'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:21.557788 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446aa327-09d9-40cb-a7d9-3dc81e30c6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:21.559354 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446aa327-09d9-40cb-a7d9-3dc81e30c6e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:21.560797 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-448bbda1-9c94-4a4a-b0a0-723176f45e82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f3329fa705e47178a6d3513eb0251d8'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:21.564395 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446aa327-09d9-40cb-a7d9-3dc81e30c6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:21.565950 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446aa327-09d9-40cb-a7d9-3dc81e30c6e0 admin admin] PATH_INFO: `/v3/projects/12ad6a8f9246427c8a4f146ec172cd1e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:21.568542 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446aa327-09d9-40cb-a7d9-3dc81e30c6e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12ad6a8f9246427c8a4f146ec172cd1e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:21.573487 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446aa327-09d9-40cb-a7d9-3dc81e30c6e0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:21.577136 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-448bbda1-9c94-4a4a-b0a0-723176f45e82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f3329fa705e47178a6d3513eb0251d8'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:21.578132 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-448bbda1-9c94-4a4a-b0a0-723176f45e82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f3329fa705e47178a6d3513eb0251d8'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:21.579928 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-448bbda1-9c94-4a4a-b0a0-723176f45e82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f3329fa705e47178a6d3513eb0251d8'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:21.656085 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1123/2272] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:16:20 2022] DELETE /identity/v3/users/4f3329fa705e47178a6d3513eb0251d8 => generated 0 bytes in 709 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:16:21.776860 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-446aa327-09d9-40cb-a7d9-3dc81e30c6e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12ad6a8f9246427c8a4f146ec172cd1e', 'request_id': 'req-446aa327-09d9-40cb-a7d9-3dc81e30c6e0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:21.802278 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-446aa327-09d9-40cb-a7d9-3dc81e30c6e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12ad6a8f9246427c8a4f146ec172cd1e', 'request_id': 'req-446aa327-09d9-40cb-a7d9-3dc81e30c6e0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:21.822460 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-446aa327-09d9-40cb-a7d9-3dc81e30c6e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12ad6a8f9246427c8a4f146ec172cd1e', 'request_id': 'req-446aa327-09d9-40cb-a7d9-3dc81e30c6e0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:21.849136 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-446aa327-09d9-40cb-a7d9-3dc81e30c6e0 admin admin] The token cache is being invalidate because project 12ad6a8f9246427c8a4f146ec172cd1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:16:21.849550 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-446aa327-09d9-40cb-a7d9-3dc81e30c6e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:21.850693 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-446aa327-09d9-40cb-a7d9-3dc81e30c6e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:21.857625 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448bbda1-9c94-4a4a-b0a0-723176f45e82 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:21.858281 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1150/2273] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:16:21 2022] DELETE /identity/v3/projects/12ad6a8f9246427c8a4f146ec172cd1e => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:16:21.859341 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-448bbda1-9c94-4a4a-b0a0-723176f45e82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:22.059087 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:22.076850 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448bbda1-9c94-4a4a-b0a0-723176f45e82 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:22.232857 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446aa327-09d9-40cb-a7d9-3dc81e30c6e0 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:22.234743 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-446aa327-09d9-40cb-a7d9-3dc81e30c6e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:22.295530 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecd46b5-cd44-4c35-8957-4c540f44bcb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:22.296514 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd46b5-cd44-4c35-8957-4c540f44bcb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:22.296969 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd46b5-cd44-4c35-8957-4c540f44bcb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:22.297422 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecd46b5-cd44-4c35-8957-4c540f44bcb0 admin admin] PATH_INFO: `/v3/users/7ea245573911407e92242319231f1c27` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:22.299053 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecd46b5-cd44-4c35-8957-4c540f44bcb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ea245573911407e92242319231f1c27)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:22.302157 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecd46b5-cd44-4c35-8957-4c540f44bcb0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:22.380786 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4ecd46b5-cd44-4c35-8957-4c540f44bcb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ea245573911407e92242319231f1c27'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:22.464346 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4ecd46b5-cd44-4c35-8957-4c540f44bcb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ea245573911407e92242319231f1c27'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:22.495952 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4ecd46b5-cd44-4c35-8957-4c540f44bcb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ea245573911407e92242319231f1c27'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:22.498079 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4ecd46b5-cd44-4c35-8957-4c540f44bcb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ea245573911407e92242319231f1c27'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:22.500281 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4ecd46b5-cd44-4c35-8957-4c540f44bcb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ea245573911407e92242319231f1c27'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:22.572934 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446aa327-09d9-40cb-a7d9-3dc81e30c6e0 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:22.591237 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0902977e-2309-458a-8364-2b6b4647b2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:22.594700 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902977e-2309-458a-8364-2b6b4647b2e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:22.596298 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902977e-2309-458a-8364-2b6b4647b2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:22.597301 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902977e-2309-458a-8364-2b6b4647b2e2 admin admin] PATH_INFO: `/v3/projects/522ec32464a14cb19b8fe3bc676827ae` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:22.607551 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0902977e-2309-458a-8364-2b6b4647b2e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=522ec32464a14cb19b8fe3bc676827ae)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:22.629394 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0902977e-2309-458a-8364-2b6b4647b2e2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:22.643479 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1124/2274] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:16:21 2022] DELETE /identity/v3/users/7ea245573911407e92242319231f1c27 => generated 0 bytes in 783 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:16:22.652895 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:22.899011 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0902977e-2309-458a-8364-2b6b4647b2e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '522ec32464a14cb19b8fe3bc676827ae', 'request_id': 'req-0902977e-2309-458a-8364-2b6b4647b2e2'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:22.930636 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0902977e-2309-458a-8364-2b6b4647b2e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '522ec32464a14cb19b8fe3bc676827ae', 'request_id': 'req-0902977e-2309-458a-8364-2b6b4647b2e2'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:22.948984 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0902977e-2309-458a-8364-2b6b4647b2e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '522ec32464a14cb19b8fe3bc676827ae', 'request_id': 'req-0902977e-2309-458a-8364-2b6b4647b2e2'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:22.987099 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0902977e-2309-458a-8364-2b6b4647b2e2 admin admin] The token cache is being invalidate because project 522ec32464a14cb19b8fe3bc676827ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:16:22.988740 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0902977e-2309-458a-8364-2b6b4647b2e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:22.989627 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-0902977e-2309-458a-8364-2b6b4647b2e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:22.999509 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1151/2275] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:16:22 2022] DELETE /identity/v3/projects/522ec32464a14cb19b8fe3bc676827ae => generated 0 bytes in 764 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:16:23.001579 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0c277-b0b1-4588-b890-2725ec84d0b6 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:23.002353 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0c277-b0b1-4588-b890-2725ec84d0b6 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:23.003045 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0c277-b0b1-4588-b890-2725ec84d0b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:23.084660 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-edc0c277-b0b1-4588-b890-2725ec84d0b6 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:16:23.103537 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-edc0c277-b0b1-4588-b890-2725ec84d0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:23.291558 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1125/2276] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:16:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:16:23.299635 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f75031-4819-49fe-afc0-348472dd9b88 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:23.299940 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f75031-4819-49fe-afc0-348472dd9b88 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:23.300219 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f75031-4819-49fe-afc0-348472dd9b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:23.362545 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c8f75031-4819-49fe-afc0-348472dd9b88 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:16:23.364104 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c8f75031-4819-49fe-afc0-348472dd9b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:23.396711 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1152/2277] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:16:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:16:23.401119 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0c277-b0b1-4588-b890-2725ec84d0b6 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:23.427183 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0c277-b0b1-4588-b890-2725ec84d0b6 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:23.446898 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0646206b-9f23-4674-9f4c-093b8e04acb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:23.446898 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0646206b-9f23-4674-9f4c-093b8e04acb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:23.446898 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0646206b-9f23-4674-9f4c-093b8e04acb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:23.446898 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0646206b-9f23-4674-9f4c-093b8e04acb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:23.448098 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0646206b-9f23-4674-9f4c-093b8e04acb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:16:23.448098 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0646206b-9f23-4674-9f4c-093b8e04acb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:23.451781 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0646206b-9f23-4674-9f4c-093b8e04acb3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:23.478088 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1126/2278] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:16:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:16:23.478767 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f75031-4819-49fe-afc0-348472dd9b88 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:23.499333 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f75031-4819-49fe-afc0-348472dd9b88 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:23.519060 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00469139-3bf3-4a65-a7ab-e14bc9ea6b40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:23.520544 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00469139-3bf3-4a65-a7ab-e14bc9ea6b40 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:23.520982 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00469139-3bf3-4a65-a7ab-e14bc9ea6b40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:23.521400 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00469139-3bf3-4a65-a7ab-e14bc9ea6b40 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:23.522284 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00469139-3bf3-4a65-a7ab-e14bc9ea6b40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:23.524653 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00469139-3bf3-4a65-a7ab-e14bc9ea6b40 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:23.586591 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1153/2279] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:16:23 2022] POST /identity/v3/projects => generated 407 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:16:23.590145 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0646206b-9f23-4674-9f4c-093b8e04acb3 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:23.611625 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0646206b-9f23-4674-9f4c-093b8e04acb3 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:23.635237 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:23.768538 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb09bd-7705-4e18-981e-455b2e4e640d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:23.769811 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb09bd-7705-4e18-981e-455b2e4e640d admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:23.770212 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb09bd-7705-4e18-981e-455b2e4e640d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:23.770602 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb09bd-7705-4e18-981e-455b2e4e640d admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:23.771702 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fb09bd-7705-4e18-981e-455b2e4e640d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:23.788225 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fb09bd-7705-4e18-981e-455b2e4e640d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:23.847316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1127/2280] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:16:23 2022] POST /identity/v3/users => generated 304 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:16:23.851567 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00469139-3bf3-4a65-a7ab-e14bc9ea6b40 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:23.868765 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00469139-3bf3-4a65-a7ab-e14bc9ea6b40 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:23.881917 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacdb6ac-571f-440f-b560-93a7411117e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:23.884555 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacdb6ac-571f-440f-b560-93a7411117e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:23.884555 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacdb6ac-571f-440f-b560-93a7411117e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:23.884555 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacdb6ac-571f-440f-b560-93a7411117e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:23.884932 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacdb6ac-571f-440f-b560-93a7411117e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:23.895837 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacdb6ac-571f-440f-b560-93a7411117e0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:23.916667 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1154/2281] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:16:23 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:16:23.920163 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb09bd-7705-4e18-981e-455b2e4e640d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:23.937233 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb09bd-7705-4e18-981e-455b2e4e640d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:23.950327 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47781fe-7a3d-49c1-ab90-ec720d7cb1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:23.951643 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47781fe-7a3d-49c1-ab90-ec720d7cb1f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:23.951936 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47781fe-7a3d-49c1-ab90-ec720d7cb1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:23.952349 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47781fe-7a3d-49c1-ab90-ec720d7cb1f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:23.953263 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47781fe-7a3d-49c1-ab90-ec720d7cb1f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:23.954753 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47781fe-7a3d-49c1-ab90-ec720d7cb1f1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:23.975214 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1128/2282] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:16:23 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:16:23.979173 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacdb6ac-571f-440f-b560-93a7411117e0 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:24.021669 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacdb6ac-571f-440f-b560-93a7411117e0 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:24.032469 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f8587-4165-4eca-8378-e8963aa8c956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:24.034524 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f8587-4165-4eca-8378-e8963aa8c956 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:24.034524 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f8587-4165-4eca-8378-e8963aa8c956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:24.034524 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f8587-4165-4eca-8378-e8963aa8c956 admin admin] PATH_INFO: `/v3/projects/fa2453d5744b4e61accf05826a2cb2bb/users/f943219fea3b493781ee9593e6dbe020/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:24.088372 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5f8587-4165-4eca-8378-e8963aa8c956 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa2453d5744b4e61accf05826a2cb2bb, user_id=f943219fea3b493781ee9593e6dbe020, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:24.091270 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5f8587-4165-4eca-8378-e8963aa8c956 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:24.128737 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1155/2283] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:16:23 2022] PUT /identity/v3/projects/fa2453d5744b4e61accf05826a2cb2bb/users/f943219fea3b493781ee9593e6dbe020/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:16:24.141042 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0789e1f2-3226-4552-bec3-76467ffc123c None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:24.142777 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0789e1f2-3226-4552-bec3-76467ffc123c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:24.143317 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0789e1f2-3226-4552-bec3-76467ffc123c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:24.182820 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-0789e1f2-3226-4552-bec3-76467ffc123c None None] MFA Rules not processed for user `f943219fea3b493781ee9593e6dbe020`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:16:24.184498 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0789e1f2-3226-4552-bec3-76467ffc123c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:24.414592 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f8587-4165-4eca-8378-e8963aa8c956 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:24.417301 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cb5f8587-4165-4eca-8378-e8963aa8c956 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:24.498947 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1129/2284] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:16:24 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:16:24.503372 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0789e1f2-3226-4552-bec3-76467ffc123c None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:24.635343 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0789e1f2-3226-4552-bec3-76467ffc123c None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:24.650882 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33684761-057c-4b60-a20d-9d6e3aa13bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:24.652371 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33684761-057c-4b60-a20d-9d6e3aa13bc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:24.654973 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33684761-057c-4b60-a20d-9d6e3aa13bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:24.655484 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33684761-057c-4b60-a20d-9d6e3aa13bc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:24.656505 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33684761-057c-4b60-a20d-9d6e3aa13bc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:24.666020 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33684761-057c-4b60-a20d-9d6e3aa13bc4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:24.699690 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f8587-4165-4eca-8378-e8963aa8c956 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:24.711572 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26215c9-bbfc-4d47-b108-1e1c37ac6710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:24.713128 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26215c9-bbfc-4d47-b108-1e1c37ac6710 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:24.714538 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26215c9-bbfc-4d47-b108-1e1c37ac6710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:24.715266 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26215c9-bbfc-4d47-b108-1e1c37ac6710 admin admin] PATH_INFO: `/v3/projects/6782c945d7464b73bf482adaff579710` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:24.719797 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26215c9-bbfc-4d47-b108-1e1c37ac6710 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6782c945d7464b73bf482adaff579710)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:24.722276 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26215c9-bbfc-4d47-b108-1e1c37ac6710 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:24.803651 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1130/2285] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:16:24 2022] POST /identity/v3/projects => generated 407 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:16:24.807974 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33684761-057c-4b60-a20d-9d6e3aa13bc4 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:24.830417 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:24.876602 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:24.919107 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c26215c9-bbfc-4d47-b108-1e1c37ac6710 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6782c945d7464b73bf482adaff579710', 'request_id': 'req-c26215c9-bbfc-4d47-b108-1e1c37ac6710'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:24.940500 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c26215c9-bbfc-4d47-b108-1e1c37ac6710 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6782c945d7464b73bf482adaff579710', 'request_id': 'req-c26215c9-bbfc-4d47-b108-1e1c37ac6710'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:24.969137 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c26215c9-bbfc-4d47-b108-1e1c37ac6710 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6782c945d7464b73bf482adaff579710', 'request_id': 'req-c26215c9-bbfc-4d47-b108-1e1c37ac6710'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:25.003867 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c26215c9-bbfc-4d47-b108-1e1c37ac6710 admin admin] The token cache is being invalidate because project 6782c945d7464b73bf482adaff579710 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:16:25.010086 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c26215c9-bbfc-4d47-b108-1e1c37ac6710 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:25.011429 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c26215c9-bbfc-4d47-b108-1e1c37ac6710 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:25.019354 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1156/2286] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:16:24 2022] DELETE /identity/v3/projects/6782c945d7464b73bf482adaff579710 => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:16:25.026597 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a824d-6bf4-41d8-aea3-eb25598bbf16 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:25.026944 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a824d-6bf4-41d8-aea3-eb25598bbf16 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:25.028673 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430a824d-6bf4-41d8-aea3-eb25598bbf16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:25.033952 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33684761-057c-4b60-a20d-9d6e3aa13bc4 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:25.116504 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-430a824d-6bf4-41d8-aea3-eb25598bbf16 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:16:25.117917 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-430a824d-6bf4-41d8-aea3-eb25598bbf16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:25.121694 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1231b290-ed2f-4f92-8ff3-3fbf08dfbe07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:25.122394 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1231b290-ed2f-4f92-8ff3-3fbf08dfbe07 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:25.122674 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1231b290-ed2f-4f92-8ff3-3fbf08dfbe07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:25.122987 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1231b290-ed2f-4f92-8ff3-3fbf08dfbe07 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:25.123689 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1231b290-ed2f-4f92-8ff3-3fbf08dfbe07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:25.140570 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1231b290-ed2f-4f92-8ff3-3fbf08dfbe07 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:25.159250 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1157/2287] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:16:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:16:25.162490 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a9ca7-d581-4b7a-be8b-b9f89ef3811c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:25.162788 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a9ca7-d581-4b7a-be8b-b9f89ef3811c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:25.163101 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a9ca7-d581-4b7a-be8b-b9f89ef3811c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:25.189038 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1131/2288] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:16:24 2022] POST /identity/v3/users => generated 310 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:16:25.190049 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1231b290-ed2f-4f92-8ff3-3fbf08dfbe07 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:25.196245 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1231b290-ed2f-4f92-8ff3-3fbf08dfbe07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:25.217877 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f60a9ca7-d581-4b7a-be8b-b9f89ef3811c None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:16:25.221205 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f60a9ca7-d581-4b7a-be8b-b9f89ef3811c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:25.225400 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1231b290-ed2f-4f92-8ff3-3fbf08dfbe07 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:25.234940 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6059ce-27de-49e6-afa3-4243ae3d91bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:25.235638 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6059ce-27de-49e6-afa3-4243ae3d91bc admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:25.235927 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6059ce-27de-49e6-afa3-4243ae3d91bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:25.236212 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6059ce-27de-49e6-afa3-4243ae3d91bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:25.236957 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6059ce-27de-49e6-afa3-4243ae3d91bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:25.246262 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1158/2289] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:16:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:16:25.246453 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6059ce-27de-49e6-afa3-4243ae3d91bc admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:25.247423 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60a9ca7-d581-4b7a-be8b-b9f89ef3811c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:25.276655 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60a9ca7-d581-4b7a-be8b-b9f89ef3811c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:25.285339 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1132/2290] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:16:25.286557 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6059ce-27de-49e6-afa3-4243ae3d91bc admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:25.295604 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf9949a-461f-4e52-988d-8759e540b39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:25.296550 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9949a-461f-4e52-988d-8759e540b39e admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:25.296844 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9949a-461f-4e52-988d-8759e540b39e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:25.297113 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9949a-461f-4e52-988d-8759e540b39e admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:25.297860 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf9949a-461f-4e52-988d-8759e540b39e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:16:25.298176 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf9949a-461f-4e52-988d-8759e540b39e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:25.299866 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf9949a-461f-4e52-988d-8759e540b39e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:25.309359 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6059ce-27de-49e6-afa3-4243ae3d91bc admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:25.325199 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1f7f6f-cc6b-42d1-ac9a-da56aedda0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:25.326119 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1f7f6f-cc6b-42d1-ac9a-da56aedda0a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:25.328258 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1f7f6f-cc6b-42d1-ac9a-da56aedda0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:25.328580 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1f7f6f-cc6b-42d1-ac9a-da56aedda0a7 admin admin] PATH_INFO: `/v3/projects/5c0bf4cfddd747218dc56693f9459fb4/users/64a65ab9438343ec980e4a9f946fb327/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:25.334700 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1159/2291] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:16:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:16:25.338044 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf9949a-461f-4e52-988d-8759e540b39e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:25.366540 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf9949a-461f-4e52-988d-8759e540b39e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:25.379098 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1f7f6f-cc6b-42d1-ac9a-da56aedda0a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c0bf4cfddd747218dc56693f9459fb4, user_id=64a65ab9438343ec980e4a9f946fb327, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:25.391102 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9fb861-8a2c-428b-82e4-22da0e9bd26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:25.392512 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1f7f6f-cc6b-42d1-ac9a-da56aedda0a7 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:25.393278 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9fb861-8a2c-428b-82e4-22da0e9bd26f admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:25.395871 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9fb861-8a2c-428b-82e4-22da0e9bd26f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:25.396258 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9fb861-8a2c-428b-82e4-22da0e9bd26f admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:25.396878 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9fb861-8a2c-428b-82e4-22da0e9bd26f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:25.400295 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9fb861-8a2c-428b-82e4-22da0e9bd26f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:25.427269 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1133/2292] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:16:25 2022] PUT /identity/v3/projects/5c0bf4cfddd747218dc56693f9459fb4/users/64a65ab9438343ec980e4a9f946fb327/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:16:25.428209 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1f7f6f-cc6b-42d1-ac9a-da56aedda0a7 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:25.509638 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1160/2293] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:16:25 2022] POST /identity/v3/projects => generated 369 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:16:25.514281 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9fb861-8a2c-428b-82e4-22da0e9bd26f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:25.744801 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1f7f6f-cc6b-42d1-ac9a-da56aedda0a7 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:25.763195 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9fb861-8a2c-428b-82e4-22da0e9bd26f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:25.767673 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d07357-4154-40dc-8686-12f47f7e0f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:25.769429 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d07357-4154-40dc-8686-12f47f7e0f53 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:25.769711 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d07357-4154-40dc-8686-12f47f7e0f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:25.770001 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d07357-4154-40dc-8686-12f47f7e0f53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:25.770729 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d07357-4154-40dc-8686-12f47f7e0f53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:25.781427 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d07357-4154-40dc-8686-12f47f7e0f53 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:25.783433 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f23caa-b357-44cc-8a70-09ad06de3d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:25.787567 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f23caa-b357-44cc-8a70-09ad06de3d06 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:25.787859 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f23caa-b357-44cc-8a70-09ad06de3d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:25.788141 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f23caa-b357-44cc-8a70-09ad06de3d06 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:25.788846 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f23caa-b357-44cc-8a70-09ad06de3d06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:25.792187 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f23caa-b357-44cc-8a70-09ad06de3d06 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:25.826324 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1134/2294] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:16:25.827385 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d07357-4154-40dc-8686-12f47f7e0f53 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:25.841836 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:25.847329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1161/2295] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:16:25 2022] POST /identity/v3/users => generated 285 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:16:25.853088 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f23caa-b357-44cc-8a70-09ad06de3d06 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:25.868791 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d07357-4154-40dc-8686-12f47f7e0f53 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:25.877064 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f23caa-b357-44cc-8a70-09ad06de3d06 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:25.884636 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a210703-156c-4023-a6f8-6769307d60ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:25.885857 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a210703-156c-4023-a6f8-6769307d60ce admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:25.886407 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a210703-156c-4023-a6f8-6769307d60ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:25.886877 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a210703-156c-4023-a6f8-6769307d60ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:25.887780 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9230b9-796d-44fc-8caa-8cfc1dce870c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:25.888186 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a210703-156c-4023-a6f8-6769307d60ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:25.889081 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9230b9-796d-44fc-8caa-8cfc1dce870c admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:25.890054 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a210703-156c-4023-a6f8-6769307d60ce admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:25.890948 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9230b9-796d-44fc-8caa-8cfc1dce870c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:25.891418 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9230b9-796d-44fc-8caa-8cfc1dce870c admin admin] PATH_INFO: `/v3/projects/5c0bf4cfddd747218dc56693f9459fb4/users/64a65ab9438343ec980e4a9f946fb327/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:25.905258 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9230b9-796d-44fc-8caa-8cfc1dce870c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c0bf4cfddd747218dc56693f9459fb4, user_id=64a65ab9438343ec980e4a9f946fb327, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:25.905774 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1162/2296] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:16:25.912977 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9230b9-796d-44fc-8caa-8cfc1dce870c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:25.919390 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a210703-156c-4023-a6f8-6769307d60ce admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:25.971431 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1135/2297] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:16:25 2022] PUT /identity/v3/projects/5c0bf4cfddd747218dc56693f9459fb4/users/64a65ab9438343ec980e4a9f946fb327/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:16:25.979198 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c1579-93bf-4b40-b9bd-04d6911a028c None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:25.979491 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c1579-93bf-4b40-b9bd-04d6911a028c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:25.979771 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4c1579-93bf-4b40-b9bd-04d6911a028c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:25.991667 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a210703-156c-4023-a6f8-6769307d60ce admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:26.048235 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ef4c1579-93bf-4b40-b9bd-04d6911a028c None None] MFA Rules not processed for user `af74115f423c43fdb2544876be4e5622`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:16:26.049733 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ef4c1579-93bf-4b40-b9bd-04d6911a028c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:26.237678 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:26.287150 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c4881-14b7-409c-9461-3f9a4eff5f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:26.287846 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c4881-14b7-409c-9461-3f9a4eff5f32 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:26.288242 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1136/2298] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:16:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:16:26.288384 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c4881-14b7-409c-9461-3f9a4eff5f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:26.289625 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3df16-68ea-492f-b273-9dda25ee41ee None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:26.290552 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3df16-68ea-492f-b273-9dda25ee41ee None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:26.290976 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3df16-68ea-492f-b273-9dda25ee41ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:26.296565 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c4881-14b7-409c-9461-3f9a4eff5f32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:26.297402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094c4881-14b7-409c-9461-3f9a4eff5f32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:26.304846 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094c4881-14b7-409c-9461-3f9a4eff5f32 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:26.331093 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1163/2299] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 422 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:16:26.332010 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c4881-14b7-409c-9461-3f9a4eff5f32 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:26.333721 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-094c4881-14b7-409c-9461-3f9a4eff5f32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:26.350626 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c4881-14b7-409c-9461-3f9a4eff5f32 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:26.356730 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfba4c2-b4f5-404a-9f6b-38a24719e65b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:26.357491 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfba4c2-b4f5-404a-9f6b-38a24719e65b admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:26.357929 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a6f3df16-68ea-492f-b273-9dda25ee41ee None None] MFA Rules not processed for user `64a65ab9438343ec980e4a9f946fb327`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:16:26.358665 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfba4c2-b4f5-404a-9f6b-38a24719e65b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:26.359060 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfba4c2-b4f5-404a-9f6b-38a24719e65b admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:26.359429 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a6f3df16-68ea-492f-b273-9dda25ee41ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:26.359900 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfba4c2-b4f5-404a-9f6b-38a24719e65b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:26.362189 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfba4c2-b4f5-404a-9f6b-38a24719e65b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:26.429236 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1164/2300] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:16:26 2022] POST /identity/v3/projects => generated 379 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:16:26.429864 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfba4c2-b4f5-404a-9f6b-38a24719e65b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:26.523155 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1137/2301] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:16:26 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:16:26.538385 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3df16-68ea-492f-b273-9dda25ee41ee None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:26.569734 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfba4c2-b4f5-404a-9f6b-38a24719e65b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:26.581818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3df16-68ea-492f-b273-9dda25ee41ee None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:26.584271 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc47d9f-a01d-4a00-b5a4-385d94ef4ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:26.585410 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc47d9f-a01d-4a00-b5a4-385d94ef4ff9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:26.585829 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc47d9f-a01d-4a00-b5a4-385d94ef4ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:26.586213 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc47d9f-a01d-4a00-b5a4-385d94ef4ff9 admin admin] PATH_INFO: `/v3/projects/d1a6a2b996b24ecd92e1783a90122fec/users/2a1e94af14e34f4197f78e9a09b8025e/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:26.600978 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb9e362-c9d3-48b9-84e4-088e3c28cda4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:26.601990 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9e362-c9d3-48b9-84e4-088e3c28cda4 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:26.602340 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9e362-c9d3-48b9-84e4-088e3c28cda4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:26.603242 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb9e362-c9d3-48b9-84e4-088e3c28cda4 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:26.604027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb9e362-c9d3-48b9-84e4-088e3c28cda4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:26.615695 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb9e362-c9d3-48b9-84e4-088e3c28cda4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:26.634646 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc47d9f-a01d-4a00-b5a4-385d94ef4ff9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1a6a2b996b24ecd92e1783a90122fec, user_id=2a1e94af14e34f4197f78e9a09b8025e, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:26.639917 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc47d9f-a01d-4a00-b5a4-385d94ef4ff9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:26.669375 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1138/2302] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:16:26 2022] POST /identity/v3/users => generated 296 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:16:26.671939 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb9e362-c9d3-48b9-84e4-088e3c28cda4 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:26.687326 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1165/2303] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:16:26 2022] PUT /identity/v3/projects/d1a6a2b996b24ecd92e1783a90122fec/users/2a1e94af14e34f4197f78e9a09b8025e/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:16:26.687530 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8eb9e362-c9d3-48b9-84e4-088e3c28cda4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:26.687614 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc47d9f-a01d-4a00-b5a4-385d94ef4ff9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:26.793347 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc47d9f-a01d-4a00-b5a4-385d94ef4ff9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:26.798637 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427eebea-9da2-4878-a00f-46c94512a0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:26.799406 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427eebea-9da2-4878-a00f-46c94512a0a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:26.799756 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427eebea-9da2-4878-a00f-46c94512a0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:26.800097 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427eebea-9da2-4878-a00f-46c94512a0a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:26.804575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427eebea-9da2-4878-a00f-46c94512a0a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:26.806647 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427eebea-9da2-4878-a00f-46c94512a0a5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:26.822690 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1166/2304] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:16:26 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:16:26.824223 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bde77f-c846-4f42-b69b-7f0f44037ad8 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:26.824586 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bde77f-c846-4f42-b69b-7f0f44037ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:26.824937 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bde77f-c846-4f42-b69b-7f0f44037ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:26.839020 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:26.849024 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb9e362-c9d3-48b9-84e4-088e3c28cda4 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:26.866136 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-31bde77f-c846-4f42-b69b-7f0f44037ad8 None None] MFA Rules not processed for user `2a1e94af14e34f4197f78e9a09b8025e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:16:26.868033 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-31bde77f-c846-4f42-b69b-7f0f44037ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:26.928854 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:26.966463 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab947ea-4ec9-4702-84a2-a33b85300878 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:26.967516 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab947ea-4ec9-4702-84a2-a33b85300878 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:26.968033 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab947ea-4ec9-4702-84a2-a33b85300878 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:26.972720 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab947ea-4ec9-4702-84a2-a33b85300878 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:26.992601 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1167/2305] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:16:26 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:16:26.992601 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bde77f-c846-4f42-b69b-7f0f44037ad8 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:27.009284 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bde77f-c846-4f42-b69b-7f0f44037ad8 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:27.014771 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2dfe37-982a-4803-8fde-5d35a582366d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:27.016071 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2dfe37-982a-4803-8fde-5d35a582366d admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:27.016357 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2dfe37-982a-4803-8fde-5d35a582366d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:27.016634 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2dfe37-982a-4803-8fde-5d35a582366d admin admin] PATH_INFO: `/v3/projects/f849da19c4384675a073acbf977b2d4a/users/ae06d907c76945e4ac189fed2b529850/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:27.035585 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2dfe37-982a-4803-8fde-5d35a582366d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f849da19c4384675a073acbf977b2d4a, user_id=ae06d907c76945e4ac189fed2b529850, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:27.038195 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2dfe37-982a-4803-8fde-5d35a582366d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:27.055113 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1168/2306] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:16:26 2022] PUT /identity/v3/projects/f849da19c4384675a073acbf977b2d4a/users/ae06d907c76945e4ac189fed2b529850/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:16:27.057158 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2dfe37-982a-4803-8fde-5d35a582366d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:27.059661 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2a2dfe37-982a-4803-8fde-5d35a582366d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:27.107756 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab947ea-4ec9-4702-84a2-a33b85300878 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:27.123303 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab947ea-4ec9-4702-84a2-a33b85300878 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:27.153812 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1139/2307] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:16:26 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:16:27.165188 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab947ea-4ec9-4702-84a2-a33b85300878 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:27.184380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2dfe37-982a-4803-8fde-5d35a582366d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:27.191575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ca1faa-4cfc-4cbd-9c03-a36b3bc0364b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1afda9d6b1d64e6ab400b26124cf1290', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:27.192239 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ca1faa-4cfc-4cbd-9c03-a36b3bc0364b service glance] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:27.192521 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ca1faa-4cfc-4cbd-9c03-a36b3bc0364b service glance] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:27.192793 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ca1faa-4cfc-4cbd-9c03-a36b3bc0364b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:27.199359 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab947ea-4ec9-4702-84a2-a33b85300878 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:27.333954 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ca1faa-4cfc-4cbd-9c03-a36b3bc0364b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:27.335630 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ca1faa-4cfc-4cbd-9c03-a36b3bc0364b service glance] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:27.348455 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1169/2308] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:16:27 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:16:27.378055 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3849da-3055-4f63-a7f8-76e71480b5af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:27.379058 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3849da-3055-4f63-a7f8-76e71480b5af admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:27.381112 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3849da-3055-4f63-a7f8-76e71480b5af admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:27.382785 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3849da-3055-4f63-a7f8-76e71480b5af admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:27.384096 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3849da-3055-4f63-a7f8-76e71480b5af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:27.387580 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3849da-3055-4f63-a7f8-76e71480b5af admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:27.419821 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1140/2309] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:16:27 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:16:27.423285 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ca1faa-4cfc-4cbd-9c03-a36b3bc0364b service glance] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:27.437906 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ca1faa-4cfc-4cbd-9c03-a36b3bc0364b service glance] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:27.448566 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffd609b-dda4-42db-a72f-349a5f9210f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:27.449908 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffd609b-dda4-42db-a72f-349a5f9210f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:27.450436 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffd609b-dda4-42db-a72f-349a5f9210f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:27.450983 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffd609b-dda4-42db-a72f-349a5f9210f2 admin admin] PATH_INFO: `/v3/projects/f849da19c4384675a073acbf977b2d4a/users/ae06d907c76945e4ac189fed2b529850/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:27.454257 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffd609b-dda4-42db-a72f-349a5f9210f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f849da19c4384675a073acbf977b2d4a, user_id=ae06d907c76945e4ac189fed2b529850, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:27.462885 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffd609b-dda4-42db-a72f-349a5f9210f2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:27.492036 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1170/2310] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:16:27 2022] PUT /identity/v3/projects/f849da19c4384675a073acbf977b2d4a/users/ae06d907c76945e4ac189fed2b529850/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:16:27.543246 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3849da-3055-4f63-a7f8-76e71480b5af admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:27.700170 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:27.712515 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3849da-3055-4f63-a7f8-76e71480b5af admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:27.908108 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963a2c22-2a9f-4ad0-a39d-bca2b1720043 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:27.908795 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a2c22-2a9f-4ad0-a39d-bca2b1720043 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:27.909053 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a2c22-2a9f-4ad0-a39d-bca2b1720043 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:27.917174 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963a2c22-2a9f-4ad0-a39d-bca2b1720043 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:27.926932 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-963a2c22-2a9f-4ad0-a39d-bca2b1720043 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:28.131249 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963a2c22-2a9f-4ad0-a39d-bca2b1720043 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:28.135072 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963a2c22-2a9f-4ad0-a39d-bca2b1720043 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:28.174595 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1141/2311] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:16:27 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 632 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:16:32.617654 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f9e06-5309-4379-bea5-751c9c9cd25e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:32.618517 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f9e06-5309-4379-bea5-751c9c9cd25e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:32.618517 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f9e06-5309-4379-bea5-751c9c9cd25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:32.682087 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-045f9e06-5309-4379-bea5-751c9c9cd25e None None] MFA Rules not processed for user `50ca9d699b004df4becfe3241802fb17`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:16:32.683315 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-045f9e06-5309-4379-bea5-751c9c9cd25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:32.735437 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:32.800721 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1171/2312] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:16:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:16:36.527651 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa04ce0c-6555-4b87-a2f4-d77d59fd4e95 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:36.528469 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa04ce0c-6555-4b87-a2f4-d77d59fd4e95 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:36.528766 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa04ce0c-6555-4b87-a2f4-d77d59fd4e95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:36.538980 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc083ed0-93f4-4331-af4e-2f100e436cf1 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:36.539540 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc083ed0-93f4-4331-af4e-2f100e436cf1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:36.539655 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc083ed0-93f4-4331-af4e-2f100e436cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:36.594921 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-cc083ed0-93f4-4331-af4e-2f100e436cf1 None None] MFA Rules not processed for user `ae06d907c76945e4ac189fed2b529850`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:16:36.596455 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cc083ed0-93f4-4331-af4e-2f100e436cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:36.603534 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-fa04ce0c-6555-4b87-a2f4-d77d59fd4e95 None None] MFA Rules not processed for user `09c23d1c732c452a9597c647ad0241f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:16:36.604968 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fa04ce0c-6555-4b87-a2f4-d77d59fd4e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:36.773169 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1172/2313] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:16:36 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:16:36.786874 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1142/2314] 10.222.0.50 () {58 vars in 1009 bytes} [Fri May 27 15:16:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:16:36.803861 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc083ed0-93f4-4331-af4e-2f100e436cf1 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:36.808356 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cc083ed0-93f4-4331-af4e-2f100e436cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:36.955602 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc083ed0-93f4-4331-af4e-2f100e436cf1 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:36.962359 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff7fe1-7354-439c-ba0c-1b696478b235 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18903f2eab7c415096e075d47f0b9a4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:36.963086 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff7fe1-7354-439c-ba0c-1b696478b235 service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:36.963376 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff7fe1-7354-439c-ba0c-1b696478b235 service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:36.963814 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff7fe1-7354-439c-ba0c-1b696478b235 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:36.974593 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ff7fe1-7354-439c-ba0c-1b696478b235 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:36.976909 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ff7fe1-7354-439c-ba0c-1b696478b235 service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:36.988368 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1173/2315] 10.222.0.50 () {60 vars in 1384 bytes} [Fri May 27 15:16:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 15:16:37.416532 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa04ce0c-6555-4b87-a2f4-d77d59fd4e95 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:37.420536 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fa04ce0c-6555-4b87-a2f4-d77d59fd4e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:37.605292 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:37.672992 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa04ce0c-6555-4b87-a2f4-d77d59fd4e95 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:37.691065 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625b45d6-453c-4fc4-aa6a-0dfc2438669e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '451a9232f95e494683d81b6078a318d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:37.691918 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b45d6-453c-4fc4-aa6a-0dfc2438669e service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:37.692207 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b45d6-453c-4fc4-aa6a-0dfc2438669e service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:37.692662 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b45d6-453c-4fc4-aa6a-0dfc2438669e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:37.851568 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625b45d6-453c-4fc4-aa6a-0dfc2438669e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:37.853276 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625b45d6-453c-4fc4-aa6a-0dfc2438669e service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:37.907311 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1143/2316] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:16:37 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:16:38.260697 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff7fe1-7354-439c-ba0c-1b696478b235 service swift] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:38.365102 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:38.529747 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff7fe1-7354-439c-ba0c-1b696478b235 service swift] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:38.563922 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cae7736-a55c-4c62-b26a-25c8631b2792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:38.571223 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae7736-a55c-4c62-b26a-25c8631b2792 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:38.574973 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae7736-a55c-4c62-b26a-25c8631b2792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:38.575346 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae7736-a55c-4c62-b26a-25c8631b2792 admin admin] PATH_INFO: `/v3/users/f943219fea3b493781ee9593e6dbe020` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:38.581782 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cae7736-a55c-4c62-b26a-25c8631b2792 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f943219fea3b493781ee9593e6dbe020)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:38.583779 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cae7736-a55c-4c62-b26a-25c8631b2792 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:38.723175 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5cae7736-a55c-4c62-b26a-25c8631b2792 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f943219fea3b493781ee9593e6dbe020'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:38.763802 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5cae7736-a55c-4c62-b26a-25c8631b2792 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f943219fea3b493781ee9593e6dbe020'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:38.766059 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5cae7736-a55c-4c62-b26a-25c8631b2792 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f943219fea3b493781ee9593e6dbe020'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:38.784878 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5cae7736-a55c-4c62-b26a-25c8631b2792 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f943219fea3b493781ee9593e6dbe020'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:38.786041 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5cae7736-a55c-4c62-b26a-25c8631b2792 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f943219fea3b493781ee9593e6dbe020'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:38.846955 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1174/2317] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:16:38 2022] DELETE /identity/v3/users/f943219fea3b493781ee9593e6dbe020 => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:16:38.848301 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd52e68-d56e-4cc0-ad66-1676684f1d75 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:38.848820 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd52e68-d56e-4cc0-ad66-1676684f1d75 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:38.848905 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd52e68-d56e-4cc0-ad66-1676684f1d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:39.019564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2fd52e68-d56e-4cc0-ad66-1676684f1d75 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:16:39.032144 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2fd52e68-d56e-4cc0-ad66-1676684f1d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:39.209716 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1144/2318] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:16:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:16:39.225693 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cae7736-a55c-4c62-b26a-25c8631b2792 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:39.227766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5cae7736-a55c-4c62-b26a-25c8631b2792 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:39.393421 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cae7736-a55c-4c62-b26a-25c8631b2792 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:39.399912 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9641ed-692c-4efc-a096-9f4f68587ece service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:39.401000 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9641ed-692c-4efc-a096-9f4f68587ece service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:39.401392 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9641ed-692c-4efc-a096-9f4f68587ece service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:39.404231 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9641ed-692c-4efc-a096-9f4f68587ece service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:39.416810 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9641ed-692c-4efc-a096-9f4f68587ece service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:39.419408 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9641ed-692c-4efc-a096-9f4f68587ece service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:39.436078 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1175/2319] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:16:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:16:40.176874 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd52e68-d56e-4cc0-ad66-1676684f1d75 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:40.179507 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2fd52e68-d56e-4cc0-ad66-1676684f1d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:40.204997 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd52e68-d56e-4cc0-ad66-1676684f1d75 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:40.222911 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e3ca5-934d-420b-8648-a6eb01fde97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:40.224063 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e3ca5-934d-420b-8648-a6eb01fde97b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:40.224351 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e3ca5-934d-420b-8648-a6eb01fde97b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:40.224709 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e3ca5-934d-420b-8648-a6eb01fde97b admin admin] PATH_INFO: `/v3/projects/fa2453d5744b4e61accf05826a2cb2bb` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:40.230813 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0e3ca5-934d-420b-8648-a6eb01fde97b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa2453d5744b4e61accf05826a2cb2bb)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:40.236131 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0e3ca5-934d-420b-8648-a6eb01fde97b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:40.310886 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4c0e3ca5-934d-420b-8648-a6eb01fde97b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa2453d5744b4e61accf05826a2cb2bb', 'request_id': 'req-4c0e3ca5-934d-420b-8648-a6eb01fde97b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:40.318632 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4c0e3ca5-934d-420b-8648-a6eb01fde97b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa2453d5744b4e61accf05826a2cb2bb', 'request_id': 'req-4c0e3ca5-934d-420b-8648-a6eb01fde97b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:40.327134 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4c0e3ca5-934d-420b-8648-a6eb01fde97b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa2453d5744b4e61accf05826a2cb2bb', 'request_id': 'req-4c0e3ca5-934d-420b-8648-a6eb01fde97b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:40.330937 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:40.357344 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4c0e3ca5-934d-420b-8648-a6eb01fde97b admin admin] The token cache is being invalidate because project fa2453d5744b4e61accf05826a2cb2bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:16:40.357683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4c0e3ca5-934d-420b-8648-a6eb01fde97b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:40.358382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4c0e3ca5-934d-420b-8648-a6eb01fde97b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:40.360572 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1145/2320] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:16:40 2022] DELETE /identity/v3/projects/fa2453d5744b4e61accf05826a2cb2bb => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:16:40.365938 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9641ed-692c-4efc-a096-9f4f68587ece service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:40.387317 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9641ed-692c-4efc-a096-9f4f68587ece service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:40.437931 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:40.522743 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6d65d8-d3ca-4115-9da6-fe6be25d888b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:40.524331 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d65d8-d3ca-4115-9da6-fe6be25d888b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:40.524716 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d65d8-d3ca-4115-9da6-fe6be25d888b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:40.525077 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6d65d8-d3ca-4115-9da6-fe6be25d888b admin admin] PATH_INFO: `/v3/users/64a65ab9438343ec980e4a9f946fb327` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:40.526356 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6d65d8-d3ca-4115-9da6-fe6be25d888b admin admin] RBAC: Authorizing `identity:delete_user(user_id=64a65ab9438343ec980e4a9f946fb327)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:40.528458 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6d65d8-d3ca-4115-9da6-fe6be25d888b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:40.623993 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1c6d65d8-d3ca-4115-9da6-fe6be25d888b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64a65ab9438343ec980e4a9f946fb327'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:40.643936 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1c6d65d8-d3ca-4115-9da6-fe6be25d888b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64a65ab9438343ec980e4a9f946fb327'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:40.646371 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1c6d65d8-d3ca-4115-9da6-fe6be25d888b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64a65ab9438343ec980e4a9f946fb327'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:40.665144 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1c6d65d8-d3ca-4115-9da6-fe6be25d888b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64a65ab9438343ec980e4a9f946fb327'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:40.666897 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1c6d65d8-d3ca-4115-9da6-fe6be25d888b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64a65ab9438343ec980e4a9f946fb327'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:40.702588 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1176/2321] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:16:40 2022] DELETE /identity/v3/users/64a65ab9438343ec980e4a9f946fb327 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:16:41.379179 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e3ca5-934d-420b-8648-a6eb01fde97b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:41.381431 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4c0e3ca5-934d-420b-8648-a6eb01fde97b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:41.409594 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e3ca5-934d-420b-8648-a6eb01fde97b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:41.678369 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203041cf-9836-493a-ae5f-c5f952743e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:41.679240 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203041cf-9836-493a-ae5f-c5f952743e82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:41.679582 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203041cf-9836-493a-ae5f-c5f952743e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:41.680418 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203041cf-9836-493a-ae5f-c5f952743e82 admin admin] PATH_INFO: `/v3/projects/5c0bf4cfddd747218dc56693f9459fb4` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:41.691240 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203041cf-9836-493a-ae5f-c5f952743e82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c0bf4cfddd747218dc56693f9459fb4)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:41.693119 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203041cf-9836-493a-ae5f-c5f952743e82 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:41.846121 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-203041cf-9836-493a-ae5f-c5f952743e82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c0bf4cfddd747218dc56693f9459fb4', 'request_id': 'req-203041cf-9836-493a-ae5f-c5f952743e82'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:41.898147 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-203041cf-9836-493a-ae5f-c5f952743e82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c0bf4cfddd747218dc56693f9459fb4', 'request_id': 'req-203041cf-9836-493a-ae5f-c5f952743e82'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:41.918894 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-203041cf-9836-493a-ae5f-c5f952743e82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c0bf4cfddd747218dc56693f9459fb4', 'request_id': 'req-203041cf-9836-493a-ae5f-c5f952743e82'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:41.973239 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-203041cf-9836-493a-ae5f-c5f952743e82 admin admin] The token cache is being invalidate because project 5c0bf4cfddd747218dc56693f9459fb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:16:41.974579 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-203041cf-9836-493a-ae5f-c5f952743e82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:41.975664 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-203041cf-9836-493a-ae5f-c5f952743e82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:16:41.984207 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1146/2322] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:16:41 2022] DELETE /identity/v3/projects/5c0bf4cfddd747218dc56693f9459fb4 => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:16:41.989770 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25045644-bd26-46ce-90a0-fe26ccbff0af None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:41.991446 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25045644-bd26-46ce-90a0-fe26ccbff0af None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:41.991874 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25045644-bd26-46ce-90a0-fe26ccbff0af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:42.115938 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-25045644-bd26-46ce-90a0-fe26ccbff0af None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:16:42.119599 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-25045644-bd26-46ce-90a0-fe26ccbff0af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:42.342543 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1177/2323] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:16:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:16:42.353038 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdee007-5521-460e-b1b4-c7b3fc78bf36 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:42.353608 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdee007-5521-460e-b1b4-c7b3fc78bf36 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:42.354112 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdee007-5521-460e-b1b4-c7b3fc78bf36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:42.411122 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-0cdee007-5521-460e-b1b4-c7b3fc78bf36 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:16:42.420836 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0cdee007-5521-460e-b1b4-c7b3fc78bf36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:42.465374 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1147/2324] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:16:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:16:42.470204 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25045644-bd26-46ce-90a0-fe26ccbff0af None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:42.490047 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25045644-bd26-46ce-90a0-fe26ccbff0af None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:42.505407 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43ac9a-d1ed-48eb-bf30-f3db55b7b4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:42.505803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43ac9a-d1ed-48eb-bf30-f3db55b7b4b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:42.506526 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43ac9a-d1ed-48eb-bf30-f3db55b7b4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:42.506526 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43ac9a-d1ed-48eb-bf30-f3db55b7b4b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:42.507427 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f43ac9a-d1ed-48eb-bf30-f3db55b7b4b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:16:42.507842 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f43ac9a-d1ed-48eb-bf30-f3db55b7b4b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:42.510167 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f43ac9a-d1ed-48eb-bf30-f3db55b7b4b4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:42.546909 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1178/2325] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:16:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:16:42.551406 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdee007-5521-460e-b1b4-c7b3fc78bf36 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:42.585500 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdee007-5521-460e-b1b4-c7b3fc78bf36 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:42.597656 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f1213-982f-4a9c-b4dd-e564e1462f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:42.600774 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f1213-982f-4a9c-b4dd-e564e1462f19 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:42.601342 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f1213-982f-4a9c-b4dd-e564e1462f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:42.602300 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f1213-982f-4a9c-b4dd-e564e1462f19 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:42.605407 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7f1213-982f-4a9c-b4dd-e564e1462f19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:42.611483 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7f1213-982f-4a9c-b4dd-e564e1462f19 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:42.678669 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1148/2326] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:16:42 2022] POST /identity/v3/projects => generated 377 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:16:42.684711 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43ac9a-d1ed-48eb-bf30-f3db55b7b4b4 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:42.704864 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43ac9a-d1ed-48eb-bf30-f3db55b7b4b4 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:42.729771 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:42.848973 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37de754e-4038-451d-9def-a336d05c0d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:42.850206 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de754e-4038-451d-9def-a336d05c0d2e admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:42.850539 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de754e-4038-451d-9def-a336d05c0d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:42.850909 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37de754e-4038-451d-9def-a336d05c0d2e admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:42.851690 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37de754e-4038-451d-9def-a336d05c0d2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:42.853768 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37de754e-4038-451d-9def-a336d05c0d2e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:42.920948 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1179/2327] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:16:42 2022] POST /identity/v3/users => generated 289 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:16:42.925171 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f1213-982f-4a9c-b4dd-e564e1462f19 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:42.942537 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f1213-982f-4a9c-b4dd-e564e1462f19 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:42.950081 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4492aa3f-dc29-405a-9a16-98e28f37de56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:42.951397 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4492aa3f-dc29-405a-9a16-98e28f37de56 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:42.951849 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4492aa3f-dc29-405a-9a16-98e28f37de56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:42.952284 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4492aa3f-dc29-405a-9a16-98e28f37de56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:42.953333 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4492aa3f-dc29-405a-9a16-98e28f37de56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:42.955362 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4492aa3f-dc29-405a-9a16-98e28f37de56 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:42.970578 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1149/2328] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:16:42 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:16:42.971847 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37de754e-4038-451d-9def-a336d05c0d2e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:42.991243 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37de754e-4038-451d-9def-a336d05c0d2e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:42.997358 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4fdd78-c61a-4fe2-80ba-e2f48cff40a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:42.998041 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4fdd78-c61a-4fe2-80ba-e2f48cff40a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:42.998324 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4fdd78-c61a-4fe2-80ba-e2f48cff40a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:42.998600 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4fdd78-c61a-4fe2-80ba-e2f48cff40a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:42.999429 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4fdd78-c61a-4fe2-80ba-e2f48cff40a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:43.001095 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4fdd78-c61a-4fe2-80ba-e2f48cff40a0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:43.017872 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4492aa3f-dc29-405a-9a16-98e28f37de56 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:43.029429 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1180/2329] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:16:42 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:16:43.035113 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4492aa3f-dc29-405a-9a16-98e28f37de56 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:43.042745 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c9511-0a10-4376-999e-6aa5da751cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:43.043809 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c9511-0a10-4376-999e-6aa5da751cec admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:43.044161 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c9511-0a10-4376-999e-6aa5da751cec admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:43.044506 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899c9511-0a10-4376-999e-6aa5da751cec admin admin] PATH_INFO: `/v3/projects/453652275b704651a4e587c99f034071/users/d3fd61a249f94f09b9a5694836b9f2e4/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:43.068213 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899c9511-0a10-4376-999e-6aa5da751cec admin admin] RBAC: Authorizing `identity:create_grant(project_id=453652275b704651a4e587c99f034071, user_id=d3fd61a249f94f09b9a5694836b9f2e4, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:43.072993 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899c9511-0a10-4376-999e-6aa5da751cec admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:43.143209 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1150/2330] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:16:43 2022] PUT /identity/v3/projects/453652275b704651a4e587c99f034071/users/d3fd61a249f94f09b9a5694836b9f2e4/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:16:43.167641 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37facf5f-1a8f-48fe-847a-2d348930d567 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:43.167641 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37facf5f-1a8f-48fe-847a-2d348930d567 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:43.167641 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37facf5f-1a8f-48fe-847a-2d348930d567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:43.213786 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-37facf5f-1a8f-48fe-847a-2d348930d567 None None] MFA Rules not processed for user `d3fd61a249f94f09b9a5694836b9f2e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:16:43.215710 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-37facf5f-1a8f-48fe-847a-2d348930d567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:43.420172 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1181/2331] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:16:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:16:43.437151 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c9511-0a10-4376-999e-6aa5da751cec admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:16:43.439589 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-899c9511-0a10-4376-999e-6aa5da751cec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:16:43.480505 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:43.546788 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899c9511-0a10-4376-999e-6aa5da751cec admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:16:43.551525 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b7ae99-d742-44a2-8923-e659ab4968be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:16:43.552190 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b7ae99-d742-44a2-8923-e659ab4968be service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:16:43.552473 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b7ae99-d742-44a2-8923-e659ab4968be service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:16:43.552806 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b7ae99-d742-44a2-8923-e659ab4968be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:16:43.563517 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b7ae99-d742-44a2-8923-e659ab4968be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:16:43.569863 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b7ae99-d742-44a2-8923-e659ab4968be service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:16:43.583199 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1151/2332] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:16:43 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:16:53.404948 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=61,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:16:53.576166 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=18 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:17:02.481325 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34db65d-3ed9-4925-b859-a0ae00a557cf None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:02.481853 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34db65d-3ed9-4925-b859-a0ae00a557cf None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:02.481853 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34db65d-3ed9-4925-b859-a0ae00a557cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:02.524296 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c34db65d-3ed9-4925-b859-a0ae00a557cf None None] MFA Rules not processed for user `09c23d1c732c452a9597c647ad0241f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:17:02.525901 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c34db65d-3ed9-4925-b859-a0ae00a557cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:02.603459 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1182/2333] 10.222.0.50 () {58 vars in 1009 bytes} [Fri May 27 15:17:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:17:02.616656 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b7ae99-d742-44a2-8923-e659ab4968be service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:02.618159 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-19b7ae99-d742-44a2-8923-e659ab4968be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:02.691815 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b7ae99-d742-44a2-8923-e659ab4968be service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:02.697004 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05e4b5f-eedc-40b5-8aeb-8f0417c5f294 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18903f2eab7c415096e075d47f0b9a4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:02.697720 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e4b5f-eedc-40b5-8aeb-8f0417c5f294 service swift] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:02.698002 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e4b5f-eedc-40b5-8aeb-8f0417c5f294 service swift] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:02.698305 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e4b5f-eedc-40b5-8aeb-8f0417c5f294 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:02.706840 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05e4b5f-eedc-40b5-8aeb-8f0417c5f294 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:02.708448 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05e4b5f-eedc-40b5-8aeb-8f0417c5f294 service swift] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:02.717474 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1152/2334] 10.222.0.50 () {60 vars in 1384 bytes} [Fri May 27 15:17:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 15:17:10.802628 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5605d8-a700-41cb-8507-ee9c24ed5ec4 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:10.803216 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5605d8-a700-41cb-8507-ee9c24ed5ec4 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:10.803216 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5605d8-a700-41cb-8507-ee9c24ed5ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:10.822125 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-ab5605d8-a700-41cb-8507-ee9c24ed5ec4 None None] MFA Rules not processed for user `09c23d1c732c452a9597c647ad0241f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:17:10.823413 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ab5605d8-a700-41cb-8507-ee9c24ed5ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:10.835823 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1183/2335] 10.222.0.50 () {58 vars in 1009 bytes} [Fri May 27 15:17:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:17:10.855827 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05e4b5f-eedc-40b5-8aeb-8f0417c5f294 service swift] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:10.866878 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05e4b5f-eedc-40b5-8aeb-8f0417c5f294 service swift] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:10.872619 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742db94e-dede-4009-8029-c10c605d382a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18903f2eab7c415096e075d47f0b9a4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:10.873594 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742db94e-dede-4009-8029-c10c605d382a service swift] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:10.874119 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742db94e-dede-4009-8029-c10c605d382a service swift] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:10.874608 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742db94e-dede-4009-8029-c10c605d382a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:10.884948 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742db94e-dede-4009-8029-c10c605d382a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:10.887126 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742db94e-dede-4009-8029-c10c605d382a service swift] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:10.897081 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1153/2336] 10.222.0.50 () {60 vars in 1384 bytes} [Fri May 27 15:17:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 15:17:11.903101 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ff20a-f118-4631-980d-9e551643f4d0 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:11.904331 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ff20a-f118-4631-980d-9e551643f4d0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:11.904627 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ff20a-f118-4631-980d-9e551643f4d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:11.962335 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-439ff20a-f118-4631-980d-9e551643f4d0 None None] MFA Rules not processed for user `af74115f423c43fdb2544876be4e5622`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:17:11.963719 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-439ff20a-f118-4631-980d-9e551643f4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:12.125660 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1184/2337] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:17:11 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:17:12.388478 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742db94e-dede-4009-8029-c10c605d382a service swift] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:12.390509 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-742db94e-dede-4009-8029-c10c605d382a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:12.622629 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742db94e-dede-4009-8029-c10c605d382a service swift] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:12.645272 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82562d3-7b5b-4d66-af3b-6b3571057c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:12.647041 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82562d3-7b5b-4d66-af3b-6b3571057c47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:12.648286 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82562d3-7b5b-4d66-af3b-6b3571057c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:12.650216 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82562d3-7b5b-4d66-af3b-6b3571057c47 admin admin] PATH_INFO: `/v3/users/2a1e94af14e34f4197f78e9a09b8025e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:12.655019 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82562d3-7b5b-4d66-af3b-6b3571057c47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a1e94af14e34f4197f78e9a09b8025e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:12.659172 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82562d3-7b5b-4d66-af3b-6b3571057c47 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:12.766748 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d82562d3-7b5b-4d66-af3b-6b3571057c47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a1e94af14e34f4197f78e9a09b8025e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:12.780388 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d82562d3-7b5b-4d66-af3b-6b3571057c47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a1e94af14e34f4197f78e9a09b8025e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:12.801792 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d82562d3-7b5b-4d66-af3b-6b3571057c47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a1e94af14e34f4197f78e9a09b8025e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:12.806708 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d82562d3-7b5b-4d66-af3b-6b3571057c47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a1e94af14e34f4197f78e9a09b8025e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:12.808341 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d82562d3-7b5b-4d66-af3b-6b3571057c47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a1e94af14e34f4197f78e9a09b8025e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:12.842383 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:17:12.866204 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1154/2338] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:17:12 2022] DELETE /identity/v3/users/2a1e94af14e34f4197f78e9a09b8025e => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:17:12.868071 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd5b76-322b-4a47-bafd-a07fb91802f9 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:12.868724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd5b76-322b-4a47-bafd-a07fb91802f9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:12.869171 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd5b76-322b-4a47-bafd-a07fb91802f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:12.917985 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b6bd5b76-322b-4a47-bafd-a07fb91802f9 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:17:12.919223 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b6bd5b76-322b-4a47-bafd-a07fb91802f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:13.071649 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1185/2339] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:17:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:17:13.086615 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82562d3-7b5b-4d66-af3b-6b3571057c47 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:13.089152 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d82562d3-7b5b-4d66-af3b-6b3571057c47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:13.177718 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82562d3-7b5b-4d66-af3b-6b3571057c47 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:13.247005 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940d5f73-42f5-4142-90eb-19e809e00bcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:13.251720 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d5f73-42f5-4142-90eb-19e809e00bcf service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:13.252637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d5f73-42f5-4142-90eb-19e809e00bcf service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:13.253090 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d5f73-42f5-4142-90eb-19e809e00bcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:13.267503 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940d5f73-42f5-4142-90eb-19e809e00bcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:13.269051 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940d5f73-42f5-4142-90eb-19e809e00bcf service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:13.288344 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1155/2340] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:17:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:17:14.023346 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd5b76-322b-4a47-bafd-a07fb91802f9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:14.023346 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b6bd5b76-322b-4a47-bafd-a07fb91802f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:14.048422 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd5b76-322b-4a47-bafd-a07fb91802f9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:14.060000 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb466e-633d-4d21-a640-0e8635eddc37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:14.060893 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb466e-633d-4d21-a640-0e8635eddc37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:14.061184 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb466e-633d-4d21-a640-0e8635eddc37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:14.061470 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb466e-633d-4d21-a640-0e8635eddc37 admin admin] PATH_INFO: `/v3/projects/d1a6a2b996b24ecd92e1783a90122fec` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:14.063057 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eb466e-633d-4d21-a640-0e8635eddc37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1a6a2b996b24ecd92e1783a90122fec)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:14.064945 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1eb466e-633d-4d21-a640-0e8635eddc37 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:14.177019 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:17:14.263678 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f1eb466e-633d-4d21-a640-0e8635eddc37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1a6a2b996b24ecd92e1783a90122fec', 'request_id': 'req-f1eb466e-633d-4d21-a640-0e8635eddc37'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:14.277754 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f1eb466e-633d-4d21-a640-0e8635eddc37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1a6a2b996b24ecd92e1783a90122fec', 'request_id': 'req-f1eb466e-633d-4d21-a640-0e8635eddc37'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:14.288817 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f1eb466e-633d-4d21-a640-0e8635eddc37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1a6a2b996b24ecd92e1783a90122fec', 'request_id': 'req-f1eb466e-633d-4d21-a640-0e8635eddc37'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:14.326512 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f1eb466e-633d-4d21-a640-0e8635eddc37 admin admin] The token cache is being invalidate because project d1a6a2b996b24ecd92e1783a90122fec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:17:14.327893 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f1eb466e-633d-4d21-a640-0e8635eddc37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:14.331560 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f1eb466e-633d-4d21-a640-0e8635eddc37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:14.339179 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1186/2341] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:17:14 2022] DELETE /identity/v3/projects/d1a6a2b996b24ecd92e1783a90122fec => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:17:14.342720 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7e0f2-8158-407d-8f53-94097b9344ca None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:14.343199 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7e0f2-8158-407d-8f53-94097b9344ca None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:14.343632 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e7e0f2-8158-407d-8f53-94097b9344ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:14.415063 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-45e7e0f2-8158-407d-8f53-94097b9344ca None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:17:14.416898 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-45e7e0f2-8158-407d-8f53-94097b9344ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:14.531654 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1156/2342] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:17:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:17:14.540082 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80bc3d-03e7-45dc-a871-af43c14e81d2 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:14.540530 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80bc3d-03e7-45dc-a871-af43c14e81d2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:14.541254 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c80bc3d-03e7-45dc-a871-af43c14e81d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:14.594595 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2c80bc3d-03e7-45dc-a871-af43c14e81d2 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:17:14.598056 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2c80bc3d-03e7-45dc-a871-af43c14e81d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:14.641542 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1187/2343] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:17:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:17:14.645689 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e7e0f2-8158-407d-8f53-94097b9344ca None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:14.662387 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e7e0f2-8158-407d-8f53-94097b9344ca None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:14.674613 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b05815-cb8a-4f03-a80e-dfb3d947d115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:14.677600 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b05815-cb8a-4f03-a80e-dfb3d947d115 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:14.677991 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b05815-cb8a-4f03-a80e-dfb3d947d115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:14.678509 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b05815-cb8a-4f03-a80e-dfb3d947d115 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:14.680437 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b05815-cb8a-4f03-a80e-dfb3d947d115 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:17:14.681766 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b05815-cb8a-4f03-a80e-dfb3d947d115 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:14.686877 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b05815-cb8a-4f03-a80e-dfb3d947d115 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:14.732793 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1157/2344] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:17:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:17:14.733888 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c80bc3d-03e7-45dc-a871-af43c14e81d2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:14.756520 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c80bc3d-03e7-45dc-a871-af43c14e81d2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:14.771295 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3477391a-345c-427b-b8df-d50b9d279dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:14.773426 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3477391a-345c-427b-b8df-d50b9d279dd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:14.774169 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3477391a-345c-427b-b8df-d50b9d279dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:14.776732 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3477391a-345c-427b-b8df-d50b9d279dd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:14.779286 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3477391a-345c-427b-b8df-d50b9d279dd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:14.784187 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3477391a-345c-427b-b8df-d50b9d279dd8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:14.845728 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1188/2345] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:17:14 2022] POST /identity/v3/projects => generated 365 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:17:14.847009 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b05815-cb8a-4f03-a80e-dfb3d947d115 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:14.861426 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b05815-cb8a-4f03-a80e-dfb3d947d115 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:15.041526 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494002cc-c437-4035-853c-97756deec949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:15.043241 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494002cc-c437-4035-853c-97756deec949 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:15.043475 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494002cc-c437-4035-853c-97756deec949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:15.043944 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494002cc-c437-4035-853c-97756deec949 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:15.044887 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494002cc-c437-4035-853c-97756deec949 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:15.047098 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494002cc-c437-4035-853c-97756deec949 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:15.097220 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1158/2346] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:17:14 2022] POST /identity/v3/users => generated 283 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:17:15.101663 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3477391a-345c-427b-b8df-d50b9d279dd8 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:15.121303 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3477391a-345c-427b-b8df-d50b9d279dd8 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:15.133562 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb276e59-4f97-4939-bd2b-86a6efdb9d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:15.134262 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb276e59-4f97-4939-bd2b-86a6efdb9d52 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:15.135841 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb276e59-4f97-4939-bd2b-86a6efdb9d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:15.136165 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb276e59-4f97-4939-bd2b-86a6efdb9d52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:15.136888 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb276e59-4f97-4939-bd2b-86a6efdb9d52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:15.140172 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb276e59-4f97-4939-bd2b-86a6efdb9d52 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:15.187594 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1189/2347] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:17:15 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:17:15.190192 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494002cc-c437-4035-853c-97756deec949 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:15.210978 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494002cc-c437-4035-853c-97756deec949 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:15.218674 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f73951-04e3-4d61-aaab-3857789c6f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:15.218674 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f73951-04e3-4d61-aaab-3857789c6f92 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:15.218674 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f73951-04e3-4d61-aaab-3857789c6f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:15.219068 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f73951-04e3-4d61-aaab-3857789c6f92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:15.219745 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f73951-04e3-4d61-aaab-3857789c6f92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:15.221480 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f73951-04e3-4d61-aaab-3857789c6f92 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:15.223418 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:17:15.249889 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1159/2348] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:17:15 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:17:15.250883 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb276e59-4f97-4939-bd2b-86a6efdb9d52 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:15.286766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb276e59-4f97-4939-bd2b-86a6efdb9d52 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:15.304399 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7fe709-1eb9-493e-805f-a3fbf88f2516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:15.305770 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7fe709-1eb9-493e-805f-a3fbf88f2516 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:15.306217 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7fe709-1eb9-493e-805f-a3fbf88f2516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:15.307806 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7fe709-1eb9-493e-805f-a3fbf88f2516 admin admin] PATH_INFO: `/v3/projects/b5bfa08f961f4f3e87eb14c4d4326d93/users/b560ccb417d34e98b3c517e74c477084/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:15.360553 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7fe709-1eb9-493e-805f-a3fbf88f2516 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5bfa08f961f4f3e87eb14c4d4326d93, user_id=b560ccb417d34e98b3c517e74c477084, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:15.369496 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7fe709-1eb9-493e-805f-a3fbf88f2516 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:15.407763 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f73951-04e3-4d61-aaab-3857789c6f92 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:15.409981 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b4f73951-04e3-4d61-aaab-3857789c6f92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:15.429332 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1190/2349] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:17:15 2022] PUT /identity/v3/projects/b5bfa08f961f4f3e87eb14c4d4326d93/users/b560ccb417d34e98b3c517e74c477084/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:17:15.440433 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48056e04-d250-4ba1-9adb-5feab2adc23b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:15.440433 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48056e04-d250-4ba1-9adb-5feab2adc23b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:15.440707 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48056e04-d250-4ba1-9adb-5feab2adc23b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:15.456009 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f73951-04e3-4d61-aaab-3857789c6f92 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:15.504489 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-48056e04-d250-4ba1-9adb-5feab2adc23b None None] MFA Rules not processed for user `b560ccb417d34e98b3c517e74c477084`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:17:15.507370 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-48056e04-d250-4ba1-9adb-5feab2adc23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:15.598628 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1191/2350] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:17:15 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:17:15.613826 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48056e04-d250-4ba1-9adb-5feab2adc23b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:15.614295 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-48056e04-d250-4ba1-9adb-5feab2adc23b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:15.631708 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8694c63e-0dd8-4c3c-ad39-876a8988ee1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:15.632687 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8694c63e-0dd8-4c3c-ad39-876a8988ee1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:15.632976 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8694c63e-0dd8-4c3c-ad39-876a8988ee1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:15.633255 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8694c63e-0dd8-4c3c-ad39-876a8988ee1d admin admin] PATH_INFO: `/v3/users/d3fd61a249f94f09b9a5694836b9f2e4` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:15.634559 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8694c63e-0dd8-4c3c-ad39-876a8988ee1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3fd61a249f94f09b9a5694836b9f2e4)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:15.638763 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8694c63e-0dd8-4c3c-ad39-876a8988ee1d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:15.736026 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:17:15.740948 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8694c63e-0dd8-4c3c-ad39-876a8988ee1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3fd61a249f94f09b9a5694836b9f2e4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:15.795903 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8694c63e-0dd8-4c3c-ad39-876a8988ee1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3fd61a249f94f09b9a5694836b9f2e4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:15.812693 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8694c63e-0dd8-4c3c-ad39-876a8988ee1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3fd61a249f94f09b9a5694836b9f2e4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:15.814044 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8694c63e-0dd8-4c3c-ad39-876a8988ee1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3fd61a249f94f09b9a5694836b9f2e4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:15.817115 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8694c63e-0dd8-4c3c-ad39-876a8988ee1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3fd61a249f94f09b9a5694836b9f2e4'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:15.823999 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48056e04-d250-4ba1-9adb-5feab2adc23b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:15.840065 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944916e4-2d6d-43df-a563-2d54e483cdaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:15.840952 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944916e4-2d6d-43df-a563-2d54e483cdaf service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:15.841695 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944916e4-2d6d-43df-a563-2d54e483cdaf service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:15.842680 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944916e4-2d6d-43df-a563-2d54e483cdaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:15.850573 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-944916e4-2d6d-43df-a563-2d54e483cdaf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:15.879758 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1160/2351] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:17:15 2022] DELETE /identity/v3/users/d3fd61a249f94f09b9a5694836b9f2e4 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:17:15.881181 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944916e4-2d6d-43df-a563-2d54e483cdaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:15.883491 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944916e4-2d6d-43df-a563-2d54e483cdaf service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:15.886505 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a304615-2797-41d3-b81b-1125c9f78eb8 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:15.886857 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a304615-2797-41d3-b81b-1125c9f78eb8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:15.887130 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a304615-2797-41d3-b81b-1125c9f78eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:15.911020 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1192/2352] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:17:15 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:17:15.944571 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8a304615-2797-41d3-b81b-1125c9f78eb8 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:17:15.948806 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8a304615-2797-41d3-b81b-1125c9f78eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:16.137043 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1161/2353] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:17:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:17:16.163930 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944916e4-2d6d-43df-a563-2d54e483cdaf service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:16.165525 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-944916e4-2d6d-43df-a563-2d54e483cdaf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:16.356988 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944916e4-2d6d-43df-a563-2d54e483cdaf service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:16.613141 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5955237b-4dcd-4c81-8ae0-58194c598b79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:16.615236 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955237b-4dcd-4c81-8ae0-58194c598b79 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:16.615525 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955237b-4dcd-4c81-8ae0-58194c598b79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:16.616321 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5955237b-4dcd-4c81-8ae0-58194c598b79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:16.636692 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5955237b-4dcd-4c81-8ae0-58194c598b79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:16.638647 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5955237b-4dcd-4c81-8ae0-58194c598b79 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:16.654971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1193/2354] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:17:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:17:17.349130 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a304615-2797-41d3-b81b-1125c9f78eb8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:17.351252 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8a304615-2797-41d3-b81b-1125c9f78eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:17.376672 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a304615-2797-41d3-b81b-1125c9f78eb8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:17.392119 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a19bcf5-bef0-4d50-bd04-695ae6cb2059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:17.393107 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19bcf5-bef0-4d50-bd04-695ae6cb2059 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:17.393388 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19bcf5-bef0-4d50-bd04-695ae6cb2059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:17.394318 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a19bcf5-bef0-4d50-bd04-695ae6cb2059 admin admin] PATH_INFO: `/v3/projects/453652275b704651a4e587c99f034071` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:17.395756 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a19bcf5-bef0-4d50-bd04-695ae6cb2059 admin admin] RBAC: Authorizing `identity:delete_project(project_id=453652275b704651a4e587c99f034071)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:17.397996 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a19bcf5-bef0-4d50-bd04-695ae6cb2059 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:17.415146 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:17:17.471922 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2a19bcf5-bef0-4d50-bd04-695ae6cb2059 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '453652275b704651a4e587c99f034071', 'request_id': 'req-2a19bcf5-bef0-4d50-bd04-695ae6cb2059'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:17.481321 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2a19bcf5-bef0-4d50-bd04-695ae6cb2059 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '453652275b704651a4e587c99f034071', 'request_id': 'req-2a19bcf5-bef0-4d50-bd04-695ae6cb2059'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:17.488071 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2a19bcf5-bef0-4d50-bd04-695ae6cb2059 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '453652275b704651a4e587c99f034071', 'request_id': 'req-2a19bcf5-bef0-4d50-bd04-695ae6cb2059'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:17.507510 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2a19bcf5-bef0-4d50-bd04-695ae6cb2059 admin admin] The token cache is being invalidate because project 453652275b704651a4e587c99f034071 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:17:17.507813 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2a19bcf5-bef0-4d50-bd04-695ae6cb2059 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:17.508547 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2a19bcf5-bef0-4d50-bd04-695ae6cb2059 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:17.512899 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1162/2355] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:17:17 2022] DELETE /identity/v3/projects/453652275b704651a4e587c99f034071 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:17:17.516502 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91fdd19-d5ae-4e8d-a55e-c83c778974e7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:17.517000 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91fdd19-d5ae-4e8d-a55e-c83c778974e7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:17.517120 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91fdd19-d5ae-4e8d-a55e-c83c778974e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:17.539938 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e91fdd19-d5ae-4e8d-a55e-c83c778974e7 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:17:17.541251 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e91fdd19-d5ae-4e8d-a55e-c83c778974e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:17.614823 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1194/2356] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:17:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:17:17.623487 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e5026-6f2b-434e-b2dc-81cbb72d083f None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:17.623778 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e5026-6f2b-434e-b2dc-81cbb72d083f None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:17.624054 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e5026-6f2b-434e-b2dc-81cbb72d083f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:17.647708 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a72e5026-6f2b-434e-b2dc-81cbb72d083f None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:17:17.648845 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a72e5026-6f2b-434e-b2dc-81cbb72d083f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:17.658017 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1163/2357] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:17:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:17:17.662237 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91fdd19-d5ae-4e8d-a55e-c83c778974e7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:17.673413 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91fdd19-d5ae-4e8d-a55e-c83c778974e7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:17.679448 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0378d6dd-3460-4310-93a1-db86d12abcb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:17.680364 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378d6dd-3460-4310-93a1-db86d12abcb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:17.680719 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378d6dd-3460-4310-93a1-db86d12abcb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:17.681071 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378d6dd-3460-4310-93a1-db86d12abcb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:17.681891 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0378d6dd-3460-4310-93a1-db86d12abcb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:17:17.682987 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0378d6dd-3460-4310-93a1-db86d12abcb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:17.684975 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0378d6dd-3460-4310-93a1-db86d12abcb3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:17.702892 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1195/2358] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:17:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:17:17.704508 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72e5026-6f2b-434e-b2dc-81cbb72d083f None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:17.714636 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72e5026-6f2b-434e-b2dc-81cbb72d083f None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:17.723231 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d38962-2ff2-4b62-b6e4-e077759c197f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:17.724014 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d38962-2ff2-4b62-b6e4-e077759c197f admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:17.724362 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d38962-2ff2-4b62-b6e4-e077759c197f admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:17.724744 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d38962-2ff2-4b62-b6e4-e077759c197f admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:17.725517 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d38962-2ff2-4b62-b6e4-e077759c197f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:17.727639 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d38962-2ff2-4b62-b6e4-e077759c197f admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:17.817502 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1164/2359] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:17:17 2022] POST /identity/v3/projects => generated 367 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:17:17.819063 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0378d6dd-3460-4310-93a1-db86d12abcb3 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:17.833930 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0378d6dd-3460-4310-93a1-db86d12abcb3 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:17.847049 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:17:17.948284 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf18df-73b6-4192-b72e-983f1dd2b304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:17.949834 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf18df-73b6-4192-b72e-983f1dd2b304 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:17.950037 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf18df-73b6-4192-b72e-983f1dd2b304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:17.950349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccf18df-73b6-4192-b72e-983f1dd2b304 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:17.951132 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccf18df-73b6-4192-b72e-983f1dd2b304 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:17.954133 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccf18df-73b6-4192-b72e-983f1dd2b304 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:17.987459 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1196/2360] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:17:17 2022] POST /identity/v3/users => generated 284 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:17:17.989520 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d38962-2ff2-4b62-b6e4-e077759c197f admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:18.047802 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d38962-2ff2-4b62-b6e4-e077759c197f admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:18.078185 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee972f67-e36a-4c76-8b82-2190ae0198ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:18.081776 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee972f67-e36a-4c76-8b82-2190ae0198ed admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:18.082335 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee972f67-e36a-4c76-8b82-2190ae0198ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:18.082871 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee972f67-e36a-4c76-8b82-2190ae0198ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:18.088824 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee972f67-e36a-4c76-8b82-2190ae0198ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:18.094617 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee972f67-e36a-4c76-8b82-2190ae0198ed admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:18.138804 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1165/2361] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:17:17 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:17:18.146023 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf18df-73b6-4192-b72e-983f1dd2b304 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:18.162886 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccf18df-73b6-4192-b72e-983f1dd2b304 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:18.171238 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3715ad6-5e3c-40be-befa-6f525a6be09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:18.172106 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3715ad6-5e3c-40be-befa-6f525a6be09c admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:18.172578 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3715ad6-5e3c-40be-befa-6f525a6be09c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:18.172952 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3715ad6-5e3c-40be-befa-6f525a6be09c admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:18.173789 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3715ad6-5e3c-40be-befa-6f525a6be09c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:18.185339 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3715ad6-5e3c-40be-befa-6f525a6be09c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:18.189213 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1197/2362] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:17:18 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:17:18.200618 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee972f67-e36a-4c76-8b82-2190ae0198ed admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:18.212987 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee972f67-e36a-4c76-8b82-2190ae0198ed admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:18.227926 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263f71bb-4a33-4f39-96a7-ec88446d356b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:18.229522 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f71bb-4a33-4f39-96a7-ec88446d356b admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:18.230162 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f71bb-4a33-4f39-96a7-ec88446d356b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:18.230517 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263f71bb-4a33-4f39-96a7-ec88446d356b admin admin] PATH_INFO: `/v3/projects/b8f2a9a46cd544ca902d88c16d7b2f98/users/09e1123fd73043838cef3fe2878ac04f/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:18.269922 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263f71bb-4a33-4f39-96a7-ec88446d356b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8f2a9a46cd544ca902d88c16d7b2f98, user_id=09e1123fd73043838cef3fe2878ac04f, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:18.277630 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263f71bb-4a33-4f39-96a7-ec88446d356b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:18.322231 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1166/2363] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:17:18 2022] PUT /identity/v3/projects/b8f2a9a46cd544ca902d88c16d7b2f98/users/09e1123fd73043838cef3fe2878ac04f/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:17:18.337178 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c25c7-81e4-48ed-a24e-1c9f1d11919d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:18.337570 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c25c7-81e4-48ed-a24e-1c9f1d11919d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:18.339067 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978c25c7-81e4-48ed-a24e-1c9f1d11919d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:18.441820 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-978c25c7-81e4-48ed-a24e-1c9f1d11919d None None] MFA Rules not processed for user `09e1123fd73043838cef3fe2878ac04f`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:17:18.451435 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-978c25c7-81e4-48ed-a24e-1c9f1d11919d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:18.641080 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1198/2364] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:17:18 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:17:18.661148 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263f71bb-4a33-4f39-96a7-ec88446d356b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:18.664705 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-263f71bb-4a33-4f39-96a7-ec88446d356b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:18.874784 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263f71bb-4a33-4f39-96a7-ec88446d356b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:18.885015 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b57e7e-9bad-4760-9a03-c8294eb9edfa service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1afda9d6b1d64e6ab400b26124cf1290', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:18.886704 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b57e7e-9bad-4760-9a03-c8294eb9edfa service glance] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:18.887025 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b57e7e-9bad-4760-9a03-c8294eb9edfa service glance] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:18.887362 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b57e7e-9bad-4760-9a03-c8294eb9edfa service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:18.910632 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b57e7e-9bad-4760-9a03-c8294eb9edfa service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:18.912527 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b57e7e-9bad-4760-9a03-c8294eb9edfa service glance] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:18.938112 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1167/2365] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:17:18 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:17:20.594974 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ca945-146e-4c1f-90c0-83d62ca0a88a None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:20.595448 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ca945-146e-4c1f-90c0-83d62ca0a88a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:20.595529 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ca945-146e-4c1f-90c0-83d62ca0a88a None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:20.597108 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1199/2366] 10.222.0.50 () {54 vars in 921 bytes} [Fri May 27 15:17:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:17:20.603999 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de614f-4e17-4d67-a290-f9a2237fa535 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:20.605082 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de614f-4e17-4d67-a290-f9a2237fa535 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:20.605523 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de614f-4e17-4d67-a290-f9a2237fa535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:20.712426 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:17:20.753541 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a8de614f-4e17-4d67-a290-f9a2237fa535 None None] MFA Rules not processed for user `451a9232f95e494683d81b6078a318d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:17:20.755749 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a8de614f-4e17-4d67-a290-f9a2237fa535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:20.942739 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1168/2367] 10.222.0.50 () {58 vars in 1018 bytes} [Fri May 27 15:17:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:17:28.615313 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=63,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:17:30.932098 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=38 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:17:32.351592 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ca945-146e-4c1f-90c0-83d62ca0a88a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:32.354507 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-469ca945-146e-4c1f-90c0-83d62ca0a88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:32.430589 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ca945-146e-4c1f-90c0-83d62ca0a88a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:32.435414 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72549e1a-2184-48cf-af15-e03a532a666b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:32.436311 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72549e1a-2184-48cf-af15-e03a532a666b service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:32.436620 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72549e1a-2184-48cf-af15-e03a532a666b service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:32.436931 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72549e1a-2184-48cf-af15-e03a532a666b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:32.438747 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-72549e1a-2184-48cf-af15-e03a532a666b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:32.533152 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72549e1a-2184-48cf-af15-e03a532a666b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:32.536265 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72549e1a-2184-48cf-af15-e03a532a666b service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:32.580783 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1200/2368] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:17:32 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:17:38.839982 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de614f-4e17-4d67-a290-f9a2237fa535 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:38.857001 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de614f-4e17-4d67-a290-f9a2237fa535 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:38.861992 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12c7d48-e62c-4d22-a34a-bdf8d9964935 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:38.863651 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c7d48-e62c-4d22-a34a-bdf8d9964935 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:38.863941 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c7d48-e62c-4d22-a34a-bdf8d9964935 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:38.864230 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c7d48-e62c-4d22-a34a-bdf8d9964935 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:38.866057 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f12c7d48-e62c-4d22-a34a-bdf8d9964935 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:39.003553 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12c7d48-e62c-4d22-a34a-bdf8d9964935 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:39.005164 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12c7d48-e62c-4d22-a34a-bdf8d9964935 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:39.016675 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1169/2369] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:17:38 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:17:39.857537 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72549e1a-2184-48cf-af15-e03a532a666b service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:39.868656 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72549e1a-2184-48cf-af15-e03a532a666b service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:39.873350 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30196793-d6af-429a-a1c3-3b9e917cfe04 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1afda9d6b1d64e6ab400b26124cf1290', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:39.874032 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30196793-d6af-429a-a1c3-3b9e917cfe04 service glance] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:39.874414 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30196793-d6af-429a-a1c3-3b9e917cfe04 service glance] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:39.874753 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30196793-d6af-429a-a1c3-3b9e917cfe04 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:39.886572 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30196793-d6af-429a-a1c3-3b9e917cfe04 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:39.888128 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30196793-d6af-429a-a1c3-3b9e917cfe04 service glance] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:39.901121 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1201/2370] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:17:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:17:40.113458 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94fc1bd-b8d9-44c4-9bf6-1dd106e3f952 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:40.114113 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94fc1bd-b8d9-44c4-9bf6-1dd106e3f952 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:40.114485 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94fc1bd-b8d9-44c4-9bf6-1dd106e3f952 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:40.116065 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1170/2371] 10.222.0.50 () {54 vars in 921 bytes} [Fri May 27 15:17:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:17:40.137993 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4a78a5-9110-4816-9f04-21f161a5c7db None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:40.138289 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4a78a5-9110-4816-9f04-21f161a5c7db None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:40.138574 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4a78a5-9110-4816-9f04-21f161a5c7db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:40.192278 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5f4a78a5-9110-4816-9f04-21f161a5c7db None None] MFA Rules not processed for user `f874edc08438434189f1d14d79291487`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:17:40.195836 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5f4a78a5-9110-4816-9f04-21f161a5c7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:40.217184 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1202/2372] 10.222.0.50 () {58 vars in 1018 bytes} [Fri May 27 15:17:40 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:17:40.237354 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94fc1bd-b8d9-44c4-9bf6-1dd106e3f952 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:40.250683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94fc1bd-b8d9-44c4-9bf6-1dd106e3f952 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:40.265023 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2efcdd-0df2-477f-8597-0d2c4ea40876 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:40.265023 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2efcdd-0df2-477f-8597-0d2c4ea40876 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:40.265023 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2efcdd-0df2-477f-8597-0d2c4ea40876 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:40.265023 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2efcdd-0df2-477f-8597-0d2c4ea40876 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:40.276134 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2efcdd-0df2-477f-8597-0d2c4ea40876 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:40.278283 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2efcdd-0df2-477f-8597-0d2c4ea40876 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:40.298003 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1171/2373] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:17:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:17:40.441339 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3c8f7-4d97-4189-a9e2-fc2bf8c38f3e None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:40.441624 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3c8f7-4d97-4189-a9e2-fc2bf8c38f3e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:40.442243 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3c8f7-4d97-4189-a9e2-fc2bf8c38f3e None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:40.445451 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1203/2374] 10.222.0.50 () {54 vars in 921 bytes} [Fri May 27 15:17:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:17:40.451405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb5bfd-209c-4bb0-932c-8530525a226e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:40.452066 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb5bfd-209c-4bb0-932c-8530525a226e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:40.452656 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bb5bfd-209c-4bb0-932c-8530525a226e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:40.487767 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-35bb5bfd-209c-4bb0-932c-8530525a226e None None] MFA Rules not processed for user `451a9232f95e494683d81b6078a318d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:17:40.489329 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-35bb5bfd-209c-4bb0-932c-8530525a226e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:40.502438 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1172/2375] 10.222.0.50 () {58 vars in 1018 bytes} [Fri May 27 15:17:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:17:40.515861 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f3c8f7-4d97-4189-a9e2-fc2bf8c38f3e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:40.536717 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f3c8f7-4d97-4189-a9e2-fc2bf8c38f3e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:40.543189 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacef686-0032-4f27-a29a-75b0f89de568 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:40.543810 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacef686-0032-4f27-a29a-75b0f89de568 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:40.544550 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacef686-0032-4f27-a29a-75b0f89de568 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:40.546669 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacef686-0032-4f27-a29a-75b0f89de568 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:40.578540 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacef686-0032-4f27-a29a-75b0f89de568 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:40.580206 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacef686-0032-4f27-a29a-75b0f89de568 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:40.598143 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1204/2376] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:17:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:17:43.262589 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a99c56-ba8b-4287-b9ac-1658d6433b0d None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:43.263223 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a99c56-ba8b-4287-b9ac-1658d6433b0d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:43.263223 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a99c56-ba8b-4287-b9ac-1658d6433b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:43.322905 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-84a99c56-ba8b-4287-b9ac-1658d6433b0d None None] MFA Rules not processed for user `09c23d1c732c452a9597c647ad0241f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:17:43.324190 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-84a99c56-ba8b-4287-b9ac-1658d6433b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:43.404623 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1173/2377] 10.222.0.50 () {58 vars in 967 bytes} [Fri May 27 15:17:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:17:43.421233 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacef686-0032-4f27-a29a-75b0f89de568 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:43.423298 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bacef686-0032-4f27-a29a-75b0f89de568 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:43.524828 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:17:43.533723 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacef686-0032-4f27-a29a-75b0f89de568 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:43.538737 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54b87c-3713-4803-a292-39f201f4ca6b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18903f2eab7c415096e075d47f0b9a4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:43.539409 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54b87c-3713-4803-a292-39f201f4ca6b service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:43.539695 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54b87c-3713-4803-a292-39f201f4ca6b service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:43.539971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54b87c-3713-4803-a292-39f201f4ca6b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:43.548609 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b54b87c-3713-4803-a292-39f201f4ca6b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:43.550200 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b54b87c-3713-4803-a292-39f201f4ca6b service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:43.558101 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1205/2378] 10.222.0.50 () {60 vars in 1384 bytes} [Fri May 27 15:17:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 15:17:45.900606 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917b6c0-a09c-47c6-9a4a-ee74f466bd6a None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:45.901177 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917b6c0-a09c-47c6-9a4a-ee74f466bd6a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:45.901434 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917b6c0-a09c-47c6-9a4a-ee74f466bd6a None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:45.903048 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1174/2379] 10.222.0.50 () {54 vars in 921 bytes} [Fri May 27 15:17:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:17:45.922181 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb556c-4f83-4e9a-a1d4-7fbb6cd16872 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:45.922729 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb556c-4f83-4e9a-a1d4-7fbb6cd16872 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:45.922984 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb556c-4f83-4e9a-a1d4-7fbb6cd16872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:45.992873 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917b6c0-a09c-47c6-9a4a-ee74f466bd6a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:45.995953 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c917b6c0-a09c-47c6-9a4a-ee74f466bd6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:45.997679 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-59eb556c-4f83-4e9a-a1d4-7fbb6cd16872 None None] MFA Rules not processed for user `8293dab3ffd647f5927d323572fdbdce`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:17:45.999464 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-59eb556c-4f83-4e9a-a1d4-7fbb6cd16872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:46.055080 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:17:46.075067 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917b6c0-a09c-47c6-9a4a-ee74f466bd6a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:46.079848 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb5dc12-cbdc-4806-af43-0feca9e9b18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:46.080799 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb5dc12-cbdc-4806-af43-0feca9e9b18b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:46.081081 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb5dc12-cbdc-4806-af43-0feca9e9b18b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:46.081358 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb5dc12-cbdc-4806-af43-0feca9e9b18b admin admin] PATH_INFO: `/v3/users/34ac2a8ad2e343838f8e3e3285ec5f55` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:46.082095 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1206/2380] 10.222.0.50 () {58 vars in 1018 bytes} [Fri May 27 15:17:45 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:17:46.082628 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb5dc12-cbdc-4806-af43-0feca9e9b18b admin admin] RBAC: Authorizing `identity:delete_user(user_id=34ac2a8ad2e343838f8e3e3285ec5f55)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:46.084671 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb5dc12-cbdc-4806-af43-0feca9e9b18b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:46.094755 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eb556c-4f83-4e9a-a1d4-7fbb6cd16872 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:46.096182 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-59eb556c-4f83-4e9a-a1d4-7fbb6cd16872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:46.110917 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eb556c-4f83-4e9a-a1d4-7fbb6cd16872 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:46.116425 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2fef6-1bc1-4167-894d-61976d176231 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8293dab3ffd647f5927d323572fdbdce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:46.117512 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2fef6-1bc1-4167-894d-61976d176231 service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:46.117873 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2fef6-1bc1-4167-894d-61976d176231 service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:46.118281 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec2fef6-1bc1-4167-894d-61976d176231 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:46.129799 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec2fef6-1bc1-4167-894d-61976d176231 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:46.131520 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec2fef6-1bc1-4167-894d-61976d176231 service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:46.141392 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-aeb5dc12-cbdc-4806-af43-0feca9e9b18b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34ac2a8ad2e343838f8e3e3285ec5f55'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:46.143270 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1207/2381] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:17:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:17:46.159796 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-aeb5dc12-cbdc-4806-af43-0feca9e9b18b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34ac2a8ad2e343838f8e3e3285ec5f55'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:46.168443 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-aeb5dc12-cbdc-4806-af43-0feca9e9b18b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34ac2a8ad2e343838f8e3e3285ec5f55'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:46.169296 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-aeb5dc12-cbdc-4806-af43-0feca9e9b18b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34ac2a8ad2e343838f8e3e3285ec5f55'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:46.170008 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-aeb5dc12-cbdc-4806-af43-0feca9e9b18b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34ac2a8ad2e343838f8e3e3285ec5f55'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:46.188293 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1175/2382] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:17:45 2022] DELETE /identity/v3/users/34ac2a8ad2e343838f8e3e3285ec5f55 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:17:46.388169 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2fef6-1bc1-4167-894d-61976d176231 service placement] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:46.401745 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec2fef6-1bc1-4167-894d-61976d176231 service placement] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:46.478620 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e678f0-8416-4520-8aad-c1d84ec3b357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:46.479724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e678f0-8416-4520-8aad-c1d84ec3b357 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:46.480149 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e678f0-8416-4520-8aad-c1d84ec3b357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:46.480455 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e678f0-8416-4520-8aad-c1d84ec3b357 admin admin] PATH_INFO: `/v3/projects/a5a48e1060474f21aacf525d26371675` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:46.481673 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e678f0-8416-4520-8aad-c1d84ec3b357 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5a48e1060474f21aacf525d26371675)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:46.483635 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e678f0-8416-4520-8aad-c1d84ec3b357 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:46.536308 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:17:46.546595 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-97e678f0-8416-4520-8aad-c1d84ec3b357 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5a48e1060474f21aacf525d26371675', 'request_id': 'req-97e678f0-8416-4520-8aad-c1d84ec3b357'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:46.559601 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-97e678f0-8416-4520-8aad-c1d84ec3b357 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5a48e1060474f21aacf525d26371675', 'request_id': 'req-97e678f0-8416-4520-8aad-c1d84ec3b357'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:46.567431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-97e678f0-8416-4520-8aad-c1d84ec3b357 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5a48e1060474f21aacf525d26371675', 'request_id': 'req-97e678f0-8416-4520-8aad-c1d84ec3b357'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:46.585203 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-97e678f0-8416-4520-8aad-c1d84ec3b357 admin admin] The token cache is being invalidate because project a5a48e1060474f21aacf525d26371675 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:17:46.585537 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-97e678f0-8416-4520-8aad-c1d84ec3b357 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:46.586244 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-97e678f0-8416-4520-8aad-c1d84ec3b357 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:17:46.588041 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1208/2383] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:17:46 2022] DELETE /identity/v3/projects/a5a48e1060474f21aacf525d26371675 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:17:46.593057 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b785edd-7876-4e61-a179-6df2f9947f3c None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:46.593678 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b785edd-7876-4e61-a179-6df2f9947f3c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:46.594028 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b785edd-7876-4e61-a179-6df2f9947f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:46.613367 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-6b785edd-7876-4e61-a179-6df2f9947f3c None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:17:46.614711 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6b785edd-7876-4e61-a179-6df2f9947f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:46.683034 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1176/2384] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:17:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:17:46.690665 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0c462-389e-44d5-aa83-8b3011be8e6b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:46.691009 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0c462-389e-44d5-aa83-8b3011be8e6b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:46.691290 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0c462-389e-44d5-aa83-8b3011be8e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:46.710569 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5cb0c462-389e-44d5-aa83-8b3011be8e6b None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:17:46.711846 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5cb0c462-389e-44d5-aa83-8b3011be8e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:46.720556 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1209/2385] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:17:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:17:46.724822 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b785edd-7876-4e61-a179-6df2f9947f3c None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:46.734610 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b785edd-7876-4e61-a179-6df2f9947f3c None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:46.739626 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a469f-5cb8-4838-b6fe-81a648b7cf47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:46.740289 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a469f-5cb8-4838-b6fe-81a648b7cf47 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:46.740572 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a469f-5cb8-4838-b6fe-81a648b7cf47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:46.740854 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a469f-5cb8-4838-b6fe-81a648b7cf47 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:46.741577 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301a469f-5cb8-4838-b6fe-81a648b7cf47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:17:46.741886 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301a469f-5cb8-4838-b6fe-81a648b7cf47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:46.744211 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301a469f-5cb8-4838-b6fe-81a648b7cf47 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:46.756361 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1177/2386] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:17:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:17:46.761496 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb0c462-389e-44d5-aa83-8b3011be8e6b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:46.770904 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb0c462-389e-44d5-aa83-8b3011be8e6b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:46.781289 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eacc51-811d-4216-8814-38a8cd85936d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:46.781984 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eacc51-811d-4216-8814-38a8cd85936d admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:46.782932 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eacc51-811d-4216-8814-38a8cd85936d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:46.782932 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eacc51-811d-4216-8814-38a8cd85936d admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:46.783632 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eacc51-811d-4216-8814-38a8cd85936d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:46.788401 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eacc51-811d-4216-8814-38a8cd85936d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:46.825226 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1210/2387] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:17:46 2022] POST /identity/v3/projects => generated 373 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:17:46.825707 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a469f-5cb8-4838-b6fe-81a648b7cf47 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:46.837128 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a469f-5cb8-4838-b6fe-81a648b7cf47 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:46.900680 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:17:46.918134 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef35348-b527-46a0-bc9b-f01a7c151e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:46.919058 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef35348-b527-46a0-bc9b-f01a7c151e88 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:46.919757 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef35348-b527-46a0-bc9b-f01a7c151e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:46.920321 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef35348-b527-46a0-bc9b-f01a7c151e88 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:46.921313 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef35348-b527-46a0-bc9b-f01a7c151e88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:46.923848 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef35348-b527-46a0-bc9b-f01a7c151e88 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:46.960842 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1178/2388] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:17:46 2022] POST /identity/v3/users => generated 287 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:17:46.965953 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eacc51-811d-4216-8814-38a8cd85936d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:46.979127 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eacc51-811d-4216-8814-38a8cd85936d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:46.982710 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b851374-1b06-4b0d-8339-6a010c5ed1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:46.983481 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b851374-1b06-4b0d-8339-6a010c5ed1d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:46.983830 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b851374-1b06-4b0d-8339-6a010c5ed1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:46.984175 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b851374-1b06-4b0d-8339-6a010c5ed1d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:46.984971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b851374-1b06-4b0d-8339-6a010c5ed1d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:46.986658 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b851374-1b06-4b0d-8339-6a010c5ed1d8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:46.998361 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1211/2389] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:17:47.002425 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef35348-b527-46a0-bc9b-f01a7c151e88 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:47.015342 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef35348-b527-46a0-bc9b-f01a7c151e88 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:47.023024 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d828132-7ccf-40ae-b1d8-3282345e7469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:47.023797 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d828132-7ccf-40ae-b1d8-3282345e7469 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:47.024145 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d828132-7ccf-40ae-b1d8-3282345e7469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:47.024486 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d828132-7ccf-40ae-b1d8-3282345e7469 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:47.025874 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d828132-7ccf-40ae-b1d8-3282345e7469 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:47.027594 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d828132-7ccf-40ae-b1d8-3282345e7469 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:47.041015 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1179/2390] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:17:47.044071 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b851374-1b06-4b0d-8339-6a010c5ed1d8 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:47.060319 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b851374-1b06-4b0d-8339-6a010c5ed1d8 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:47.068207 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb43d5-6008-418d-a06e-ff4d8a915207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:47.069303 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb43d5-6008-418d-a06e-ff4d8a915207 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:47.069629 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb43d5-6008-418d-a06e-ff4d8a915207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:47.069944 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfb43d5-6008-418d-a06e-ff4d8a915207 admin admin] PATH_INFO: `/v3/projects/d83b2967950d49838b8a38bcdb367fb6/users/1983aae8f70741f1a6c28c513236cbff/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:47.089841 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfb43d5-6008-418d-a06e-ff4d8a915207 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d83b2967950d49838b8a38bcdb367fb6, user_id=1983aae8f70741f1a6c28c513236cbff, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:47.092577 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfb43d5-6008-418d-a06e-ff4d8a915207 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:47.112567 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1212/2391] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:17:47 2022] PUT /identity/v3/projects/d83b2967950d49838b8a38bcdb367fb6/users/1983aae8f70741f1a6c28c513236cbff/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:17:47.122820 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a9a54-73f3-446d-9a7d-4e8f20cf3883 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:47.123173 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a9a54-73f3-446d-9a7d-4e8f20cf3883 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:47.123512 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a9a54-73f3-446d-9a7d-4e8f20cf3883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:47.158998 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-cd3a9a54-73f3-446d-9a7d-4e8f20cf3883 None None] MFA Rules not processed for user `1983aae8f70741f1a6c28c513236cbff`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:17:47.160329 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cd3a9a54-73f3-446d-9a7d-4e8f20cf3883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:47.232542 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1180/2392] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:17:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:17:47.244101 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb43d5-6008-418d-a06e-ff4d8a915207 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:47.246124 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4bfb43d5-6008-418d-a06e-ff4d8a915207 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:47.315853 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfb43d5-6008-418d-a06e-ff4d8a915207 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:47.319573 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da42e25-251d-4cb9-9ba9-ab7d45a41141 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:47.320222 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da42e25-251d-4cb9-9ba9-ab7d45a41141 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:47.320532 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da42e25-251d-4cb9-9ba9-ab7d45a41141 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:47.320780 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da42e25-251d-4cb9-9ba9-ab7d45a41141 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:47.330477 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da42e25-251d-4cb9-9ba9-ab7d45a41141 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:47.331871 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da42e25-251d-4cb9-9ba9-ab7d45a41141 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:47.340882 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1213/2393] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:17:47 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:17:51.655410 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a25bd-60f8-45fe-8e66-db449cde7209 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:51.656131 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a25bd-60f8-45fe-8e66-db449cde7209 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:51.656131 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0a25bd-60f8-45fe-8e66-db449cde7209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:51.675339 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-8b0a25bd-60f8-45fe-8e66-db449cde7209 None None] MFA Rules not processed for user `09c23d1c732c452a9597c647ad0241f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:17:51.676462 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8b0a25bd-60f8-45fe-8e66-db449cde7209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:51.735686 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1181/2394] 10.222.0.50 () {58 vars in 1009 bytes} [Fri May 27 15:17:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:17:51.748629 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da42e25-251d-4cb9-9ba9-ab7d45a41141 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:17:51.750050 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1da42e25-251d-4cb9-9ba9-ab7d45a41141 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:51.820358 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da42e25-251d-4cb9-9ba9-ab7d45a41141 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:17:51.826064 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7256d2e8-3fc8-4d6c-8a7d-cc04d38e10b4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18903f2eab7c415096e075d47f0b9a4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:17:51.826765 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256d2e8-3fc8-4d6c-8a7d-cc04d38e10b4 service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:51.827016 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256d2e8-3fc8-4d6c-8a7d-cc04d38e10b4 service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:51.827372 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256d2e8-3fc8-4d6c-8a7d-cc04d38e10b4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:51.836241 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7256d2e8-3fc8-4d6c-8a7d-cc04d38e10b4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:17:51.839089 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:17:51.842451 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7256d2e8-3fc8-4d6c-8a7d-cc04d38e10b4 service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:17:51.853125 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1214/2395] 10.222.0.50 () {60 vars in 1384 bytes} [Fri May 27 15:17:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 15:17:56.297568 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27eb03-0f01-4f49-89cb-b102d2e11472 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:17:56.298094 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27eb03-0f01-4f49-89cb-b102d2e11472 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:17:56.298452 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27eb03-0f01-4f49-89cb-b102d2e11472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:17:56.323725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-0c27eb03-0f01-4f49-89cb-b102d2e11472 None None] MFA Rules not processed for user `9c9149cb2b3c423796164b487a305bbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:17:56.329473 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0c27eb03-0f01-4f49-89cb-b102d2e11472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:17:56.407090 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1182/2396] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:17:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:01.855692 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:03.938512 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7256d2e8-3fc8-4d6c-8a7d-cc04d38e10b4 service swift] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:03.940104 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7256d2e8-3fc8-4d6c-8a7d-cc04d38e10b4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:04.015297 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7256d2e8-3fc8-4d6c-8a7d-cc04d38e10b4 service swift] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:04.020984 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9673dea5-b239-4d55-9c3e-cf0feb5d138c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8293dab3ffd647f5927d323572fdbdce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:04.021634 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673dea5-b239-4d55-9c3e-cf0feb5d138c service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:04.021917 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673dea5-b239-4d55-9c3e-cf0feb5d138c service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:04.022192 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673dea5-b239-4d55-9c3e-cf0feb5d138c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:04.024118 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9673dea5-b239-4d55-9c3e-cf0feb5d138c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:04.036357 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9673dea5-b239-4d55-9c3e-cf0feb5d138c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:04.037913 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9673dea5-b239-4d55-9c3e-cf0feb5d138c service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:04.049227 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1215/2397] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:18:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:18:04.733329 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c27eb03-0f01-4f49-89cb-b102d2e11472 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:04.746260 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:04.796868 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c27eb03-0f01-4f49-89cb-b102d2e11472 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:04.801538 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff55d313-9baa-4900-945e-b6979969e6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:04.802398 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff55d313-9baa-4900-945e-b6979969e6e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:04.802677 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff55d313-9baa-4900-945e-b6979969e6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:04.802986 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff55d313-9baa-4900-945e-b6979969e6e6 admin admin] PATH_INFO: `/v3/users/1983aae8f70741f1a6c28c513236cbff` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:04.804167 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff55d313-9baa-4900-945e-b6979969e6e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1983aae8f70741f1a6c28c513236cbff)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:04.806122 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff55d313-9baa-4900-945e-b6979969e6e6 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:04.852091 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ff55d313-9baa-4900-945e-b6979969e6e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1983aae8f70741f1a6c28c513236cbff'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:04.859743 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ff55d313-9baa-4900-945e-b6979969e6e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1983aae8f70741f1a6c28c513236cbff'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:04.866600 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ff55d313-9baa-4900-945e-b6979969e6e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1983aae8f70741f1a6c28c513236cbff'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:04.867251 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ff55d313-9baa-4900-945e-b6979969e6e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1983aae8f70741f1a6c28c513236cbff'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:04.867863 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ff55d313-9baa-4900-945e-b6979969e6e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1983aae8f70741f1a6c28c513236cbff'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:04.889577 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1183/2398] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:18:04 2022] DELETE /identity/v3/users/1983aae8f70741f1a6c28c513236cbff => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:04.894248 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07882f73-a0d3-4205-b5bc-241702af4e3c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:04.894540 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07882f73-a0d3-4205-b5bc-241702af4e3c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:04.894819 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07882f73-a0d3-4205-b5bc-241702af4e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:04.911647 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-07882f73-a0d3-4205-b5bc-241702af4e3c None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:04.912830 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-07882f73-a0d3-4205-b5bc-241702af4e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:04.973902 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1216/2399] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:18:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:04.984144 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff55d313-9baa-4900-945e-b6979969e6e6 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:04.985603 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ff55d313-9baa-4900-945e-b6979969e6e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:05.061090 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff55d313-9baa-4900-945e-b6979969e6e6 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:05.065595 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8135b42-2513-4183-9a8c-228ada3de019 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:05.065917 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8135b42-2513-4183-9a8c-228ada3de019 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:05.066295 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8135b42-2513-4183-9a8c-228ada3de019 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:05.066668 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8135b42-2513-4183-9a8c-228ada3de019 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:05.082487 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8135b42-2513-4183-9a8c-228ada3de019 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:05.087146 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8135b42-2513-4183-9a8c-228ada3de019 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:05.099113 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1184/2400] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:18:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:18:05.484690 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07882f73-a0d3-4205-b5bc-241702af4e3c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:05.486110 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-07882f73-a0d3-4205-b5bc-241702af4e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:05.498314 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07882f73-a0d3-4205-b5bc-241702af4e3c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:05.503033 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4e28e7-47de-45a0-9002-8415a09a00ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:05.503966 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e28e7-47de-45a0-9002-8415a09a00ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:05.504320 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e28e7-47de-45a0-9002-8415a09a00ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:05.504662 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e28e7-47de-45a0-9002-8415a09a00ae admin admin] PATH_INFO: `/v3/projects/d83b2967950d49838b8a38bcdb367fb6` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:05.505900 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4e28e7-47de-45a0-9002-8415a09a00ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=d83b2967950d49838b8a38bcdb367fb6)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:05.508789 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4e28e7-47de-45a0-9002-8415a09a00ae admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:05.560904 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-aa4e28e7-47de-45a0-9002-8415a09a00ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd83b2967950d49838b8a38bcdb367fb6', 'request_id': 'req-aa4e28e7-47de-45a0-9002-8415a09a00ae'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:05.568373 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-aa4e28e7-47de-45a0-9002-8415a09a00ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd83b2967950d49838b8a38bcdb367fb6', 'request_id': 'req-aa4e28e7-47de-45a0-9002-8415a09a00ae'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:05.574343 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-aa4e28e7-47de-45a0-9002-8415a09a00ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd83b2967950d49838b8a38bcdb367fb6', 'request_id': 'req-aa4e28e7-47de-45a0-9002-8415a09a00ae'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:05.585715 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-aa4e28e7-47de-45a0-9002-8415a09a00ae admin admin] The token cache is being invalidate because project d83b2967950d49838b8a38bcdb367fb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:18:05.586006 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-aa4e28e7-47de-45a0-9002-8415a09a00ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:05.586655 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-aa4e28e7-47de-45a0-9002-8415a09a00ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:05.588773 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1217/2401] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:18:05 2022] DELETE /identity/v3/projects/d83b2967950d49838b8a38bcdb367fb6 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:18:05.595043 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eeb486-0e9f-4904-a399-d4242ed330ac None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:05.595341 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eeb486-0e9f-4904-a399-d4242ed330ac None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:05.595624 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eeb486-0e9f-4904-a399-d4242ed330ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:05.614939 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a7eeb486-0e9f-4904-a399-d4242ed330ac None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:05.616186 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a7eeb486-0e9f-4904-a399-d4242ed330ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:05.647855 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:05.683738 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1185/2402] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:18:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:05.691237 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3540023-6000-4a94-8948-e6a4b958a79d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:05.691527 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3540023-6000-4a94-8948-e6a4b958a79d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:05.691792 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3540023-6000-4a94-8948-e6a4b958a79d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:05.716311 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c3540023-6000-4a94-8948-e6a4b958a79d None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:05.719540 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c3540023-6000-4a94-8948-e6a4b958a79d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:05.732800 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1218/2403] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:18:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:05.737963 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eeb486-0e9f-4904-a399-d4242ed330ac None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:05.749798 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eeb486-0e9f-4904-a399-d4242ed330ac None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:05.754980 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e39d69c-20d5-4f21-8d13-caef47cbe264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:05.755662 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e39d69c-20d5-4f21-8d13-caef47cbe264 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:05.755941 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e39d69c-20d5-4f21-8d13-caef47cbe264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:05.756328 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e39d69c-20d5-4f21-8d13-caef47cbe264 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:05.757071 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e39d69c-20d5-4f21-8d13-caef47cbe264 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:18:05.757424 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e39d69c-20d5-4f21-8d13-caef47cbe264 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:05.759060 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e39d69c-20d5-4f21-8d13-caef47cbe264 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:05.777668 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1186/2404] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:18:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:18:05.779119 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3540023-6000-4a94-8948-e6a4b958a79d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:05.789557 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3540023-6000-4a94-8948-e6a4b958a79d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:05.796078 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e459ca66-d28f-420f-8bc9-f37d1813ccf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:05.796839 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e459ca66-d28f-420f-8bc9-f37d1813ccf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:05.797267 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e459ca66-d28f-420f-8bc9-f37d1813ccf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:05.797562 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e459ca66-d28f-420f-8bc9-f37d1813ccf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:05.798465 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e459ca66-d28f-420f-8bc9-f37d1813ccf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:05.810891 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e459ca66-d28f-420f-8bc9-f37d1813ccf5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:05.815219 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:05.848034 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1219/2405] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:18:05 2022] POST /identity/v3/projects => generated 373 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:18:05.852110 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e39d69c-20d5-4f21-8d13-caef47cbe264 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:05.862077 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e39d69c-20d5-4f21-8d13-caef47cbe264 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:05.918775 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc196b7-0bd5-4bbe-9cec-5f9aafc3eac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:05.919617 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc196b7-0bd5-4bbe-9cec-5f9aafc3eac4 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:05.919900 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc196b7-0bd5-4bbe-9cec-5f9aafc3eac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:05.920177 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc196b7-0bd5-4bbe-9cec-5f9aafc3eac4 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:05.920866 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc196b7-0bd5-4bbe-9cec-5f9aafc3eac4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:05.922820 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc196b7-0bd5-4bbe-9cec-5f9aafc3eac4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:05.943934 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1187/2406] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:18:05 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:18:05.948945 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e459ca66-d28f-420f-8bc9-f37d1813ccf5 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:05.959588 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e459ca66-d28f-420f-8bc9-f37d1813ccf5 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:05.965739 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4268ed04-5382-43a6-bc8d-4e6e17e1bf04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:05.966876 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268ed04-5382-43a6-bc8d-4e6e17e1bf04 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:05.966876 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268ed04-5382-43a6-bc8d-4e6e17e1bf04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:05.967178 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268ed04-5382-43a6-bc8d-4e6e17e1bf04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:05.967908 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4268ed04-5382-43a6-bc8d-4e6e17e1bf04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:05.969780 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4268ed04-5382-43a6-bc8d-4e6e17e1bf04 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:05.980424 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1220/2407] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:18:05.984504 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc196b7-0bd5-4bbe-9cec-5f9aafc3eac4 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:05.995215 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc196b7-0bd5-4bbe-9cec-5f9aafc3eac4 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:06.000034 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b363db5-e4bf-4022-8fb4-5cdfd4349b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:06.001110 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b363db5-e4bf-4022-8fb4-5cdfd4349b45 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:06.001399 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b363db5-e4bf-4022-8fb4-5cdfd4349b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:06.001679 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b363db5-e4bf-4022-8fb4-5cdfd4349b45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:06.002406 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b363db5-e4bf-4022-8fb4-5cdfd4349b45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:06.004054 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b363db5-e4bf-4022-8fb4-5cdfd4349b45 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:06.014718 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1188/2408] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:18:06.019259 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4268ed04-5382-43a6-bc8d-4e6e17e1bf04 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:06.029493 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4268ed04-5382-43a6-bc8d-4e6e17e1bf04 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:06.034101 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21439fb-3619-448b-b7ce-d5673f3a429e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:06.035015 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21439fb-3619-448b-b7ce-d5673f3a429e admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:06.035298 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21439fb-3619-448b-b7ce-d5673f3a429e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:06.035569 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21439fb-3619-448b-b7ce-d5673f3a429e admin admin] PATH_INFO: `/v3/projects/520d6685a60d4d9b8b7415d11b01a8f0/users/f52a9eabb6f84931b8f0af715af9eb36/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:06.051946 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21439fb-3619-448b-b7ce-d5673f3a429e admin admin] RBAC: Authorizing `identity:create_grant(project_id=520d6685a60d4d9b8b7415d11b01a8f0, user_id=f52a9eabb6f84931b8f0af715af9eb36, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:06.055032 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21439fb-3619-448b-b7ce-d5673f3a429e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:06.069916 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1221/2409] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:18:06 2022] PUT /identity/v3/projects/520d6685a60d4d9b8b7415d11b01a8f0/users/f52a9eabb6f84931b8f0af715af9eb36/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:06.076074 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b524d8-229d-4ae8-974b-1ebd1633b6bd None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:06.076571 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b524d8-229d-4ae8-974b-1ebd1633b6bd None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:06.076913 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b524d8-229d-4ae8-974b-1ebd1633b6bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:06.099069 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-68b524d8-229d-4ae8-974b-1ebd1633b6bd None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:06.100550 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-68b524d8-229d-4ae8-974b-1ebd1633b6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:06.209548 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1189/2410] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:18:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:06.220786 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21439fb-3619-448b-b7ce-d5673f3a429e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:06.222647 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d21439fb-3619-448b-b7ce-d5673f3a429e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:06.294570 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21439fb-3619-448b-b7ce-d5673f3a429e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:06.298611 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbfdf40-d3c9-4008-9fc5-05c9f85a768d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:06.300113 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfdf40-d3c9-4008-9fc5-05c9f85a768d service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:06.300631 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfdf40-d3c9-4008-9fc5-05c9f85a768d service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:06.300742 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfdf40-d3c9-4008-9fc5-05c9f85a768d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:06.317911 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbfdf40-d3c9-4008-9fc5-05c9f85a768d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:06.319614 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbfdf40-d3c9-4008-9fc5-05c9f85a768d service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:06.343940 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1222/2411] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:18:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:18:06.885958 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b524d8-229d-4ae8-974b-1ebd1633b6bd None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:06.887586 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-68b524d8-229d-4ae8-974b-1ebd1633b6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:06.902789 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b524d8-229d-4ae8-974b-1ebd1633b6bd None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:06.909208 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d39b5b-faf7-4e91-bb1d-32a2d60d2eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:06.910529 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d39b5b-faf7-4e91-bb1d-32a2d60d2eca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:06.911006 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d39b5b-faf7-4e91-bb1d-32a2d60d2eca admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:06.911424 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d39b5b-faf7-4e91-bb1d-32a2d60d2eca admin admin] PATH_INFO: `/v3/users/09e1123fd73043838cef3fe2878ac04f` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:06.912994 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d39b5b-faf7-4e91-bb1d-32a2d60d2eca admin admin] RBAC: Authorizing `identity:delete_user(user_id=09e1123fd73043838cef3fe2878ac04f)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:06.915614 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d39b5b-faf7-4e91-bb1d-32a2d60d2eca admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:06.980942 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:06.992336 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-00d39b5b-faf7-4e91-bb1d-32a2d60d2eca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09e1123fd73043838cef3fe2878ac04f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:07.003066 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-00d39b5b-faf7-4e91-bb1d-32a2d60d2eca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09e1123fd73043838cef3fe2878ac04f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:07.017749 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-00d39b5b-faf7-4e91-bb1d-32a2d60d2eca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09e1123fd73043838cef3fe2878ac04f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:07.019560 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-00d39b5b-faf7-4e91-bb1d-32a2d60d2eca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09e1123fd73043838cef3fe2878ac04f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:07.020274 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-00d39b5b-faf7-4e91-bb1d-32a2d60d2eca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09e1123fd73043838cef3fe2878ac04f'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:07.062879 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1190/2412] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:18:06 2022] DELETE /identity/v3/users/09e1123fd73043838cef3fe2878ac04f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:07.063814 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962195c8-9d98-4dc1-b343-ecac22c49bcd None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:07.064292 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962195c8-9d98-4dc1-b343-ecac22c49bcd None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:07.064397 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962195c8-9d98-4dc1-b343-ecac22c49bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:07.097569 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-962195c8-9d98-4dc1-b343-ecac22c49bcd None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:07.098792 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-962195c8-9d98-4dc1-b343-ecac22c49bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:07.229303 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1223/2413] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:18:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:07.248698 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d39b5b-faf7-4e91-bb1d-32a2d60d2eca admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:07.250362 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-00d39b5b-faf7-4e91-bb1d-32a2d60d2eca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:07.332423 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d39b5b-faf7-4e91-bb1d-32a2d60d2eca admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:07.336819 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc97fcb3-5763-49bb-9fbe-975180bb2950 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:07.337623 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97fcb3-5763-49bb-9fbe-975180bb2950 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:07.338209 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97fcb3-5763-49bb-9fbe-975180bb2950 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:07.338576 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97fcb3-5763-49bb-9fbe-975180bb2950 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:07.349914 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc97fcb3-5763-49bb-9fbe-975180bb2950 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:07.351507 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc97fcb3-5763-49bb-9fbe-975180bb2950 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:07.365046 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1191/2414] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:18:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:18:07.668287 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962195c8-9d98-4dc1-b343-ecac22c49bcd None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:07.669884 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-962195c8-9d98-4dc1-b343-ecac22c49bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:07.718892 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962195c8-9d98-4dc1-b343-ecac22c49bcd None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:07.759587 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32d18c-08e4-4dc8-aef3-40da9c48377d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:07.765493 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32d18c-08e4-4dc8-aef3-40da9c48377d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:07.765786 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32d18c-08e4-4dc8-aef3-40da9c48377d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:07.766083 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32d18c-08e4-4dc8-aef3-40da9c48377d admin admin] PATH_INFO: `/v3/projects/b8f2a9a46cd544ca902d88c16d7b2f98` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:07.769879 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a32d18c-08e4-4dc8-aef3-40da9c48377d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8f2a9a46cd544ca902d88c16d7b2f98)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:07.772152 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a32d18c-08e4-4dc8-aef3-40da9c48377d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:07.872516 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:07.882323 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1a32d18c-08e4-4dc8-aef3-40da9c48377d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8f2a9a46cd544ca902d88c16d7b2f98', 'request_id': 'req-1a32d18c-08e4-4dc8-aef3-40da9c48377d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:07.897381 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1a32d18c-08e4-4dc8-aef3-40da9c48377d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8f2a9a46cd544ca902d88c16d7b2f98', 'request_id': 'req-1a32d18c-08e4-4dc8-aef3-40da9c48377d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:07.907840 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1a32d18c-08e4-4dc8-aef3-40da9c48377d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8f2a9a46cd544ca902d88c16d7b2f98', 'request_id': 'req-1a32d18c-08e4-4dc8-aef3-40da9c48377d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:07.929648 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1a32d18c-08e4-4dc8-aef3-40da9c48377d admin admin] The token cache is being invalidate because project b8f2a9a46cd544ca902d88c16d7b2f98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:18:07.929945 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1a32d18c-08e4-4dc8-aef3-40da9c48377d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:07.934505 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1a32d18c-08e4-4dc8-aef3-40da9c48377d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:07.939485 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1224/2415] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:18:07 2022] DELETE /identity/v3/projects/b8f2a9a46cd544ca902d88c16d7b2f98 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:11.833825 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc0753-1d23-4abd-803c-74e0ebf639f8 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:11.834424 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc0753-1d23-4abd-803c-74e0ebf639f8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:11.834809 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc0753-1d23-4abd-803c-74e0ebf639f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:11.873324 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-6ebc0753-1d23-4abd-803c-74e0ebf639f8 None None] MFA Rules not processed for user `f52a9eabb6f84931b8f0af715af9eb36`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:11.874611 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6ebc0753-1d23-4abd-803c-74e0ebf639f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:11.952972 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1192/2416] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:18:11 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:11.957565 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32d18c-08e4-4dc8-aef3-40da9c48377d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:11.959351 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1a32d18c-08e4-4dc8-aef3-40da9c48377d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:12.035132 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a32d18c-08e4-4dc8-aef3-40da9c48377d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:12.039768 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d315d7-57f9-4901-bf36-5d38fb6bf350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:12.040444 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d315d7-57f9-4901-bf36-5d38fb6bf350 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:12.040725 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d315d7-57f9-4901-bf36-5d38fb6bf350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:12.041724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d315d7-57f9-4901-bf36-5d38fb6bf350 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:12.042421 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d315d7-57f9-4901-bf36-5d38fb6bf350 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:12.044402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d315d7-57f9-4901-bf36-5d38fb6bf350 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:12.070679 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1225/2417] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:18:11 2022] POST /identity/v3/projects => generated 371 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:18:12.075141 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebc0753-1d23-4abd-803c-74e0ebf639f8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:12.092178 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebc0753-1d23-4abd-803c-74e0ebf639f8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:12.143599 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:12.168379 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ec4e69-57fe-4c3b-9077-1ba5f95e7929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:12.169194 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec4e69-57fe-4c3b-9077-1ba5f95e7929 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:12.169635 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec4e69-57fe-4c3b-9077-1ba5f95e7929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:12.169989 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ec4e69-57fe-4c3b-9077-1ba5f95e7929 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:12.170992 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ec4e69-57fe-4c3b-9077-1ba5f95e7929 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:12.173226 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ec4e69-57fe-4c3b-9077-1ba5f95e7929 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:12.195261 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1193/2418] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:18:12 2022] POST /identity/v3/users => generated 292 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:18:12.199314 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d315d7-57f9-4901-bf36-5d38fb6bf350 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:12.209619 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d315d7-57f9-4901-bf36-5d38fb6bf350 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:12.214828 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2256c6b2-d014-47c9-96cf-cd3736fa8c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:12.215520 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2256c6b2-d014-47c9-96cf-cd3736fa8c2f admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:12.215806 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2256c6b2-d014-47c9-96cf-cd3736fa8c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:12.216084 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2256c6b2-d014-47c9-96cf-cd3736fa8c2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:12.216803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2256c6b2-d014-47c9-96cf-cd3736fa8c2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:12.218413 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2256c6b2-d014-47c9-96cf-cd3736fa8c2f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:12.231834 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1226/2419] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:18:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:18:12.235300 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ec4e69-57fe-4c3b-9077-1ba5f95e7929 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:12.245528 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ec4e69-57fe-4c3b-9077-1ba5f95e7929 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:12.251418 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277b651a-4c78-46a1-8d37-8ab45b865d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:12.252429 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b651a-4c78-46a1-8d37-8ab45b865d6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:12.252791 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b651a-4c78-46a1-8d37-8ab45b865d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:12.253077 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b651a-4c78-46a1-8d37-8ab45b865d6d admin admin] PATH_INFO: `/v3/projects/010f41373a55498ab8a43a15609778de/users/94926e51a8954fd8a836eba351a176de/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:12.269992 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277b651a-4c78-46a1-8d37-8ab45b865d6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=010f41373a55498ab8a43a15609778de, user_id=94926e51a8954fd8a836eba351a176de, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:12.272700 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277b651a-4c78-46a1-8d37-8ab45b865d6d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:12.287247 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1194/2420] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:18:12 2022] PUT /identity/v3/projects/010f41373a55498ab8a43a15609778de/users/94926e51a8954fd8a836eba351a176de/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:12.291799 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2256c6b2-d014-47c9-96cf-cd3736fa8c2f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:12.301875 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2256c6b2-d014-47c9-96cf-cd3736fa8c2f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:12.366742 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a977f2-f0be-40a1-a070-9e2998d4a4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:12.367456 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a977f2-f0be-40a1-a070-9e2998d4a4c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:12.367743 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a977f2-f0be-40a1-a070-9e2998d4a4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:12.368026 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a977f2-f0be-40a1-a070-9e2998d4a4c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:12.368764 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a977f2-f0be-40a1-a070-9e2998d4a4c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:12.371118 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a977f2-f0be-40a1-a070-9e2998d4a4c2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:12.382255 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1227/2421] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:18:12 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:18:12.387339 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277b651a-4c78-46a1-8d37-8ab45b865d6d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:12.397846 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277b651a-4c78-46a1-8d37-8ab45b865d6d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:12.402592 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d915b5f8-0d64-46e3-90c8-3c9451209ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:12.403935 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d915b5f8-0d64-46e3-90c8-3c9451209ee1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:12.404182 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d915b5f8-0d64-46e3-90c8-3c9451209ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:12.404455 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d915b5f8-0d64-46e3-90c8-3c9451209ee1 admin admin] PATH_INFO: `/v3/projects/010f41373a55498ab8a43a15609778de/users/94926e51a8954fd8a836eba351a176de/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:12.406454 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d915b5f8-0d64-46e3-90c8-3c9451209ee1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=010f41373a55498ab8a43a15609778de, user_id=94926e51a8954fd8a836eba351a176de, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:12.409229 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d915b5f8-0d64-46e3-90c8-3c9451209ee1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:12.422746 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1195/2422] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:18:12 2022] PUT /identity/v3/projects/010f41373a55498ab8a43a15609778de/users/94926e51a8954fd8a836eba351a176de/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:12.466978 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a977f2-f0be-40a1-a070-9e2998d4a4c2 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:12.469052 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d9a977f2-f0be-40a1-a070-9e2998d4a4c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:12.482549 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a977f2-f0be-40a1-a070-9e2998d4a4c2 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:12.546098 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:12.573229 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ab1954-fad1-4c3f-b95c-000a341d95a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:12.574539 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab1954-fad1-4c3f-b95c-000a341d95a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:12.575072 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab1954-fad1-4c3f-b95c-000a341d95a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:12.575360 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab1954-fad1-4c3f-b95c-000a341d95a7 admin admin] PATH_INFO: `/v3/users/b560ccb417d34e98b3c517e74c477084` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:12.578414 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ab1954-fad1-4c3f-b95c-000a341d95a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b560ccb417d34e98b3c517e74c477084)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:12.581006 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ab1954-fad1-4c3f-b95c-000a341d95a7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:12.659196 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-71ab1954-fad1-4c3f-b95c-000a341d95a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b560ccb417d34e98b3c517e74c477084'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:12.668200 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-71ab1954-fad1-4c3f-b95c-000a341d95a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b560ccb417d34e98b3c517e74c477084'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:12.668966 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-71ab1954-fad1-4c3f-b95c-000a341d95a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b560ccb417d34e98b3c517e74c477084'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:12.680272 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-71ab1954-fad1-4c3f-b95c-000a341d95a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b560ccb417d34e98b3c517e74c477084'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:12.681769 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-71ab1954-fad1-4c3f-b95c-000a341d95a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b560ccb417d34e98b3c517e74c477084'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:12.709894 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1228/2423] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:18:12 2022] DELETE /identity/v3/users/b560ccb417d34e98b3c517e74c477084 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:12.711581 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5e283-ce64-4f18-a02f-5b028ec4b826 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:12.712049 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5e283-ce64-4f18-a02f-5b028ec4b826 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:12.712382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5e283-ce64-4f18-a02f-5b028ec4b826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:12.733297 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-0df5e283-ce64-4f18-a02f-5b028ec4b826 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:12.734731 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0df5e283-ce64-4f18-a02f-5b028ec4b826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:12.815667 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1196/2424] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:18:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:12.828521 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ab1954-fad1-4c3f-b95c-000a341d95a7 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:12.830181 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-71ab1954-fad1-4c3f-b95c-000a341d95a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:12.916076 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ab1954-fad1-4c3f-b95c-000a341d95a7 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:12.922041 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e963a9-fa6f-4073-a727-490268c312b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:12.922715 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e963a9-fa6f-4073-a727-490268c312b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:12.923041 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e963a9-fa6f-4073-a727-490268c312b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:12.923319 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e963a9-fa6f-4073-a727-490268c312b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:12.935359 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e963a9-fa6f-4073-a727-490268c312b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:12.936901 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e963a9-fa6f-4073-a727-490268c312b3 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:12.952324 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1229/2425] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:18:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:18:13.407719 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5e283-ce64-4f18-a02f-5b028ec4b826 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:13.409496 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0df5e283-ce64-4f18-a02f-5b028ec4b826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:13.429672 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5e283-ce64-4f18-a02f-5b028ec4b826 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:13.435607 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba98a3d-1ed7-4dca-ad29-6a64a2dd6bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:13.437077 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba98a3d-1ed7-4dca-ad29-6a64a2dd6bf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:13.437370 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba98a3d-1ed7-4dca-ad29-6a64a2dd6bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:13.437643 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba98a3d-1ed7-4dca-ad29-6a64a2dd6bf5 admin admin] PATH_INFO: `/v3/projects/b5bfa08f961f4f3e87eb14c4d4326d93` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:13.439270 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba98a3d-1ed7-4dca-ad29-6a64a2dd6bf5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5bfa08f961f4f3e87eb14c4d4326d93)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:13.441191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba98a3d-1ed7-4dca-ad29-6a64a2dd6bf5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:13.496546 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-cba98a3d-1ed7-4dca-ad29-6a64a2dd6bf5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5bfa08f961f4f3e87eb14c4d4326d93', 'request_id': 'req-cba98a3d-1ed7-4dca-ad29-6a64a2dd6bf5'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:13.505225 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-cba98a3d-1ed7-4dca-ad29-6a64a2dd6bf5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5bfa08f961f4f3e87eb14c4d4326d93', 'request_id': 'req-cba98a3d-1ed7-4dca-ad29-6a64a2dd6bf5'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:13.512650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-cba98a3d-1ed7-4dca-ad29-6a64a2dd6bf5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5bfa08f961f4f3e87eb14c4d4326d93', 'request_id': 'req-cba98a3d-1ed7-4dca-ad29-6a64a2dd6bf5'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:13.529607 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-cba98a3d-1ed7-4dca-ad29-6a64a2dd6bf5 admin admin] The token cache is being invalidate because project b5bfa08f961f4f3e87eb14c4d4326d93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:18:13.529966 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-cba98a3d-1ed7-4dca-ad29-6a64a2dd6bf5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:13.530999 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-cba98a3d-1ed7-4dca-ad29-6a64a2dd6bf5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:18:13.533733 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1197/2426] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:18:13 2022] DELETE /identity/v3/projects/b5bfa08f961f4f3e87eb14c4d4326d93 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:18:13.538754 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f322f912-d329-43e9-906c-b6be346af3ca None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:13.539818 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f322f912-d329-43e9-906c-b6be346af3ca None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:13.540257 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f322f912-d329-43e9-906c-b6be346af3ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:13.561846 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f322f912-d329-43e9-906c-b6be346af3ca None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:13.563607 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f322f912-d329-43e9-906c-b6be346af3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:13.649997 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1230/2427] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:18:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:13.652524 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabff07a-44ab-4860-a5e7-85bc6f4dea45 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:13.653379 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabff07a-44ab-4860-a5e7-85bc6f4dea45 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:13.653761 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabff07a-44ab-4860-a5e7-85bc6f4dea45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:13.675568 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-cabff07a-44ab-4860-a5e7-85bc6f4dea45 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:13.676871 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cabff07a-44ab-4860-a5e7-85bc6f4dea45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:13.688347 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1198/2428] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:18:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:13.689596 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f322f912-d329-43e9-906c-b6be346af3ca None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:13.701411 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f322f912-d329-43e9-906c-b6be346af3ca None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:13.707051 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ccdf52-22c1-4f74-a8ae-6ca7b17618e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:13.707828 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ccdf52-22c1-4f74-a8ae-6ca7b17618e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:13.708180 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ccdf52-22c1-4f74-a8ae-6ca7b17618e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:13.708561 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ccdf52-22c1-4f74-a8ae-6ca7b17618e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:13.709356 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ccdf52-22c1-4f74-a8ae-6ca7b17618e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:18:13.709728 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ccdf52-22c1-4f74-a8ae-6ca7b17618e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:13.711464 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ccdf52-22c1-4f74-a8ae-6ca7b17618e9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:13.715860 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:13.733484 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabff07a-44ab-4860-a5e7-85bc6f4dea45 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:13.735035 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1231/2429] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:18:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:18:13.743326 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabff07a-44ab-4860-a5e7-85bc6f4dea45 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:13.748725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ce713-cac2-4545-907a-246baa00e9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:13.749389 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ce713-cac2-4545-907a-246baa00e9e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:13.749672 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ce713-cac2-4545-907a-246baa00e9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:13.749948 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ce713-cac2-4545-907a-246baa00e9e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:13.750637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8ce713-cac2-4545-907a-246baa00e9e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:13.753425 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8ce713-cac2-4545-907a-246baa00e9e8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:13.765334 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:13.790282 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1199/2430] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:18:13 2022] POST /identity/v3/projects => generated 381 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:18:13.795320 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ccdf52-22c1-4f74-a8ae-6ca7b17618e9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:13.806165 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ccdf52-22c1-4f74-a8ae-6ca7b17618e9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:13.867435 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c23c961-da9d-46df-894d-8991b86309c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:13.868465 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23c961-da9d-46df-894d-8991b86309c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:13.869533 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23c961-da9d-46df-894d-8991b86309c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:13.870013 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c23c961-da9d-46df-894d-8991b86309c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:13.870947 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c23c961-da9d-46df-894d-8991b86309c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:13.873086 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c23c961-da9d-46df-894d-8991b86309c1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:13.895050 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1232/2431] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:18:13 2022] POST /identity/v3/users => generated 291 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:18:13.899263 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ce713-cac2-4545-907a-246baa00e9e8 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:13.911089 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ce713-cac2-4545-907a-246baa00e9e8 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:13.916630 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efb663a-0fd6-4a87-9fd7-e55f9b0b616d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:13.917375 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb663a-0fd6-4a87-9fd7-e55f9b0b616d admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:13.918543 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb663a-0fd6-4a87-9fd7-e55f9b0b616d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:13.918543 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb663a-0fd6-4a87-9fd7-e55f9b0b616d admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:13.918728 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efb663a-0fd6-4a87-9fd7-e55f9b0b616d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:13.920555 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efb663a-0fd6-4a87-9fd7-e55f9b0b616d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:13.935109 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1200/2432] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:18:13.939450 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c23c961-da9d-46df-894d-8991b86309c1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:13.957630 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c23c961-da9d-46df-894d-8991b86309c1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:13.967691 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d07db2-89e8-4be1-a908-e37a42799cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:13.969213 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d07db2-89e8-4be1-a908-e37a42799cb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:13.969806 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d07db2-89e8-4be1-a908-e37a42799cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:13.970203 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d07db2-89e8-4be1-a908-e37a42799cb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:13.971137 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d07db2-89e8-4be1-a908-e37a42799cb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:13.973443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d07db2-89e8-4be1-a908-e37a42799cb9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:13.986681 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1233/2433] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:18:13 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:18:13.990804 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efb663a-0fd6-4a87-9fd7-e55f9b0b616d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:14.005729 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efb663a-0fd6-4a87-9fd7-e55f9b0b616d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:14.011305 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb49c56-d511-43b1-b569-bde1bf7e3423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:14.012191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb49c56-d511-43b1-b569-bde1bf7e3423 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:14.012532 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb49c56-d511-43b1-b569-bde1bf7e3423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:14.012813 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb49c56-d511-43b1-b569-bde1bf7e3423 admin admin] PATH_INFO: `/v3/projects/58318fdd40dd469099dade09793f6b38/users/0b257d6c8a034981be9b54789c313d3c/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:14.029288 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb49c56-d511-43b1-b569-bde1bf7e3423 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58318fdd40dd469099dade09793f6b38, user_id=0b257d6c8a034981be9b54789c313d3c, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:14.032016 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb49c56-d511-43b1-b569-bde1bf7e3423 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:14.047268 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1201/2434] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:18:13 2022] PUT /identity/v3/projects/58318fdd40dd469099dade09793f6b38/users/0b257d6c8a034981be9b54789c313d3c/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:18:14.052341 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ba72d-f0e8-4f05-b311-292e11c13c41 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:14.052638 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ba72d-f0e8-4f05-b311-292e11c13c41 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:14.052915 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ba72d-f0e8-4f05-b311-292e11c13c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:14.074980 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-111ba72d-f0e8-4f05-b311-292e11c13c41 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:14.076770 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-111ba72d-f0e8-4f05-b311-292e11c13c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:14.176968 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1234/2435] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:18:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:14.186695 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb49c56-d511-43b1-b569-bde1bf7e3423 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:14.188581 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-acb49c56-d511-43b1-b569-bde1bf7e3423 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:14.281209 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb49c56-d511-43b1-b569-bde1bf7e3423 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:14.287595 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dc634e-aa2c-4f47-8d02-f2079862972e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:14.288868 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc634e-aa2c-4f47-8d02-f2079862972e service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:14.289349 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc634e-aa2c-4f47-8d02-f2079862972e service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:14.289830 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dc634e-aa2c-4f47-8d02-f2079862972e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:14.306453 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dc634e-aa2c-4f47-8d02-f2079862972e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:14.308794 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dc634e-aa2c-4f47-8d02-f2079862972e service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:14.353967 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1202/2436] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:18:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:18:15.824165 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b973bd-6602-46a6-9937-270c5cfcc036 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:15.824687 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b973bd-6602-46a6-9937-270c5cfcc036 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:15.824814 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b973bd-6602-46a6-9937-270c5cfcc036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:15.904021 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a4b973bd-6602-46a6-9937-270c5cfcc036 None None] MFA Rules not processed for user `0b257d6c8a034981be9b54789c313d3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:15.905398 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a4b973bd-6602-46a6-9937-270c5cfcc036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:16.086596 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1235/2437] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:18:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:16.100784 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dc634e-aa2c-4f47-8d02-f2079862972e service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:16.104619 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-54dc634e-aa2c-4f47-8d02-f2079862972e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:16.214512 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dc634e-aa2c-4f47-8d02-f2079862972e service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:16.218039 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef4d487-6bc0-4a91-9d45-003609cca361 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:16.219362 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4d487-6bc0-4a91-9d45-003609cca361 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:16.219362 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4d487-6bc0-4a91-9d45-003609cca361 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:16.219553 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4d487-6bc0-4a91-9d45-003609cca361 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:16.251774 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef4d487-6bc0-4a91-9d45-003609cca361 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:16.253257 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef4d487-6bc0-4a91-9d45-003609cca361 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:16.284977 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1203/2438] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:18:16 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:18:18.050733 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f59a0-a110-4965-ae36-384e5e464475 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:18.051830 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f59a0-a110-4965-ae36-384e5e464475 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:18.052536 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f59a0-a110-4965-ae36-384e5e464475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:18.060576 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:18.089605 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-193f59a0-a110-4965-ae36-384e5e464475 None None] MFA Rules not processed for user `0b257d6c8a034981be9b54789c313d3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:18.091675 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-193f59a0-a110-4965-ae36-384e5e464475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:18.117616 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1236/2439] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:18:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:18.809426 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b58b862-b08e-4458-81ac-b6b2ddf27cd3 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:18.810316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b58b862-b08e-4458-81ac-b6b2ddf27cd3 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:18.810748 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b58b862-b08e-4458-81ac-b6b2ddf27cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:18.868998 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3b58b862-b08e-4458-81ac-b6b2ddf27cd3 None None] MFA Rules not processed for user `94926e51a8954fd8a836eba351a176de`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:18.870500 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3b58b862-b08e-4458-81ac-b6b2ddf27cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:18.985399 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:19.030885 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1204/2440] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:18:18 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:19.100763 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f59a0-a110-4965-ae36-384e5e464475 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:19.102292 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-193f59a0-a110-4965-ae36-384e5e464475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:19.248716 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f59a0-a110-4965-ae36-384e5e464475 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:19.255140 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966f9cb-af28-4989-a5b1-9fc5d0644cd4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '451a9232f95e494683d81b6078a318d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:19.269329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966f9cb-af28-4989-a5b1-9fc5d0644cd4 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:19.269751 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966f9cb-af28-4989-a5b1-9fc5d0644cd4 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:19.270043 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966f9cb-af28-4989-a5b1-9fc5d0644cd4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:19.276545 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9966f9cb-af28-4989-a5b1-9fc5d0644cd4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:19.395695 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9966f9cb-af28-4989-a5b1-9fc5d0644cd4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:19.397663 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9966f9cb-af28-4989-a5b1-9fc5d0644cd4 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:19.409752 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1237/2441] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:18:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:18:21.178825 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165212a8-9259-4960-a789-619b069a79c1 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:21.179415 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165212a8-9259-4960-a789-619b069a79c1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:21.179567 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165212a8-9259-4960-a789-619b069a79c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:21.204320 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-165212a8-9259-4960-a789-619b069a79c1 None None] MFA Rules not processed for user `f52a9eabb6f84931b8f0af715af9eb36`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:21.205670 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-165212a8-9259-4960-a789-619b069a79c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:21.213547 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1205/2442] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:18:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:18:22.869716 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ec1a9-ba41-439e-a2a9-d11d4830d7f7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:22.870220 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ec1a9-ba41-439e-a2a9-d11d4830d7f7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:22.873403 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9ec1a9-ba41-439e-a2a9-d11d4830d7f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:22.929983 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2d9ec1a9-ba41-439e-a2a9-d11d4830d7f7 None None] MFA Rules not processed for user `09c23d1c732c452a9597c647ad0241f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:18:22.935663 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2d9ec1a9-ba41-439e-a2a9-d11d4830d7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:23.097616 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1238/2443] 10.222.0.50 () {58 vars in 967 bytes} [Fri May 27 15:18:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:18:23.123052 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165212a8-9259-4960-a789-619b069a79c1 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:18:23.124628 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-165212a8-9259-4960-a789-619b069a79c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:18:23.218031 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165212a8-9259-4960-a789-619b069a79c1 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:18:23.223968 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836522ff-8e71-426f-bc69-b6f8ed2f5759 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18903f2eab7c415096e075d47f0b9a4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:18:23.224922 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836522ff-8e71-426f-bc69-b6f8ed2f5759 service swift] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:18:23.225369 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836522ff-8e71-426f-bc69-b6f8ed2f5759 service swift] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:18:23.225816 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836522ff-8e71-426f-bc69-b6f8ed2f5759 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:18:23.235970 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836522ff-8e71-426f-bc69-b6f8ed2f5759 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:18:23.237640 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836522ff-8e71-426f-bc69-b6f8ed2f5759 service swift] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:18:23.247684 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1206/2444] 10.222.0.50 () {60 vars in 1384 bytes} [Fri May 27 15:18:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 15:18:33.089763 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=82 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:18:33.243951 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=41 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:19:01.932934 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9ec1a9-ba41-439e-a2a9-d11d4830d7f7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:01.942677 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9ec1a9-ba41-439e-a2a9-d11d4830d7f7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:01.946354 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3a17eb-520c-4eba-a165-daad60476df1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:01.947022 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a17eb-520c-4eba-a165-daad60476df1 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:01.947310 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a17eb-520c-4eba-a165-daad60476df1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:01.947590 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a17eb-520c-4eba-a165-daad60476df1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:01.958516 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3a17eb-520c-4eba-a165-daad60476df1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:01.960857 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3a17eb-520c-4eba-a165-daad60476df1 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:01.971742 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1239/2445] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:19:01 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:19:11.966655 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:19:35.941983 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7822e063-9a36-4251-bc98-33d8f2cf7bc4 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:35.942592 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7822e063-9a36-4251-bc98-33d8f2cf7bc4 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:35.942592 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7822e063-9a36-4251-bc98-33d8f2cf7bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:35.960202 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7822e063-9a36-4251-bc98-33d8f2cf7bc4 None None] MFA Rules not processed for user `0b257d6c8a034981be9b54789c313d3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:19:35.961416 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7822e063-9a36-4251-bc98-33d8f2cf7bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:35.968585 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1207/2446] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:19:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:19:41.643299 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3a17eb-520c-4eba-a165-daad60476df1 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:41.651790 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3a17eb-520c-4eba-a165-daad60476df1 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:41.655434 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae45daf-e48e-4b45-b6be-3d095e1159e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:41.656074 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae45daf-e48e-4b45-b6be-3d095e1159e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:41.656358 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae45daf-e48e-4b45-b6be-3d095e1159e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:41.656632 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae45daf-e48e-4b45-b6be-3d095e1159e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:41.659292 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6ae45daf-e48e-4b45-b6be-3d095e1159e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:41.673208 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae45daf-e48e-4b45-b6be-3d095e1159e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:41.675230 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae45daf-e48e-4b45-b6be-3d095e1159e8 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:41.686416 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1240/2447] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:19:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:19:42.271562 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7822e063-9a36-4251-bc98-33d8f2cf7bc4 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:42.281666 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7822e063-9a36-4251-bc98-33d8f2cf7bc4 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:42.288431 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a32f38-350b-4830-aeb4-a0394002100c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:42.289992 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a32f38-350b-4830-aeb4-a0394002100c service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:42.290550 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a32f38-350b-4830-aeb4-a0394002100c service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:42.291006 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a32f38-350b-4830-aeb4-a0394002100c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:42.293937 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c6a32f38-350b-4830-aeb4-a0394002100c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:42.311164 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a32f38-350b-4830-aeb4-a0394002100c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:42.313208 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a32f38-350b-4830-aeb4-a0394002100c service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:42.326396 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1208/2448] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:19:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:19:48.780446 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d21e4f-0b32-42a7-910c-4c2349c43479 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:48.781030 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d21e4f-0b32-42a7-910c-4c2349c43479 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:48.781319 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d21e4f-0b32-42a7-910c-4c2349c43479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:48.801944 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b6d21e4f-0b32-42a7-910c-4c2349c43479 None None] MFA Rules not processed for user `5a990918de6d409a949616da90de6a5e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:19:48.803293 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b6d21e4f-0b32-42a7-910c-4c2349c43479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:48.877822 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1241/2449] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:19:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:19:49.617882 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a32f38-350b-4830-aeb4-a0394002100c service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:49.620697 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c6a32f38-350b-4830-aeb4-a0394002100c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:49.709126 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a32f38-350b-4830-aeb4-a0394002100c service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:49.715036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d4bd54-bdf5-41af-ba66-ad57de9c319d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8293dab3ffd647f5927d323572fdbdce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:49.715591 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4bd54-bdf5-41af-ba66-ad57de9c319d service placement] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:49.715875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4bd54-bdf5-41af-ba66-ad57de9c319d service placement] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:49.716153 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4bd54-bdf5-41af-ba66-ad57de9c319d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:49.717938 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a1d4bd54-bdf5-41af-ba66-ad57de9c319d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:49.730454 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d4bd54-bdf5-41af-ba66-ad57de9c319d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:49.732073 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d4bd54-bdf5-41af-ba66-ad57de9c319d service placement] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:49.742599 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1209/2450] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:19:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:19:54.610502 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d21e4f-0b32-42a7-910c-4c2349c43479 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:54.612994 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b6d21e4f-0b32-42a7-910c-4c2349c43479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:54.625205 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d21e4f-0b32-42a7-910c-4c2349c43479 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:54.633375 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2cf91c-b9f3-4027-9433-c86ac7732de5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:54.634526 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2cf91c-b9f3-4027-9433-c86ac7732de5 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:54.634944 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2cf91c-b9f3-4027-9433-c86ac7732de5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:54.635713 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2cf91c-b9f3-4027-9433-c86ac7732de5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:54.637794 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9a2cf91c-b9f3-4027-9433-c86ac7732de5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:54.652694 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2cf91c-b9f3-4027-9433-c86ac7732de5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:54.655032 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2cf91c-b9f3-4027-9433-c86ac7732de5 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:54.666128 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1242/2451] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:19:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:19:56.468453 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d4bd54-bdf5-41af-ba66-ad57de9c319d service placement] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:56.482535 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d4bd54-bdf5-41af-ba66-ad57de9c319d service placement] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:56.486899 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beaf0c8-9710-443b-8666-c1c07c8004f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:56.487531 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beaf0c8-9710-443b-8666-c1c07c8004f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:56.487820 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beaf0c8-9710-443b-8666-c1c07c8004f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:56.488151 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1beaf0c8-9710-443b-8666-c1c07c8004f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:56.489897 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1beaf0c8-9710-443b-8666-c1c07c8004f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:56.558551 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beaf0c8-9710-443b-8666-c1c07c8004f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:56.560333 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1beaf0c8-9710-443b-8666-c1c07c8004f5 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:56.571730 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1210/2452] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:19:56 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:19:58.318823 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd4e22-67a2-48bd-98db-0ba826c98247 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:58.319387 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd4e22-67a2-48bd-98db-0ba826c98247 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:58.319555 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd4e22-67a2-48bd-98db-0ba826c98247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:58.345988 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1edd4e22-67a2-48bd-98db-0ba826c98247 None None] MFA Rules not processed for user `387048f42b634693a7e827a8d88ef89b`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:19:58.347931 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1edd4e22-67a2-48bd-98db-0ba826c98247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:58.359561 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1243/2453] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:19:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:19:58.823621 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beaf0c8-9710-443b-8666-c1c07c8004f5 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:58.836014 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1beaf0c8-9710-443b-8666-c1c07c8004f5 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:58.842604 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36286569-b8db-49e5-bd70-d628aeb7185d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:58.843303 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36286569-b8db-49e5-bd70-d628aeb7185d service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:58.843570 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36286569-b8db-49e5-bd70-d628aeb7185d service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:58.843853 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36286569-b8db-49e5-bd70-d628aeb7185d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:58.846302 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-36286569-b8db-49e5-bd70-d628aeb7185d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:58.858491 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36286569-b8db-49e5-bd70-d628aeb7185d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:58.861424 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36286569-b8db-49e5-bd70-d628aeb7185d service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:58.877649 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1211/2454] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:19:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:19:59.031819 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edd4e22-67a2-48bd-98db-0ba826c98247 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:19:59.033540 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1edd4e22-67a2-48bd-98db-0ba826c98247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:59.054179 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edd4e22-67a2-48bd-98db-0ba826c98247 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:19:59.067190 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33d0a5c-925e-4019-8404-6dd60f41b765 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '451a9232f95e494683d81b6078a318d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:19:59.068385 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d0a5c-925e-4019-8404-6dd60f41b765 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:19:59.068817 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d0a5c-925e-4019-8404-6dd60f41b765 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:19:59.069238 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33d0a5c-925e-4019-8404-6dd60f41b765 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:19:59.071559 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c33d0a5c-925e-4019-8404-6dd60f41b765 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:19:59.086312 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33d0a5c-925e-4019-8404-6dd60f41b765 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:19:59.089502 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33d0a5c-925e-4019-8404-6dd60f41b765 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:19:59.107438 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1244/2455] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:19:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:20:00.219913 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b104c7-a5d2-4c18-9fcf-3dbf5aa72ede None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:00.221191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b104c7-a5d2-4c18-9fcf-3dbf5aa72ede None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:00.221191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b104c7-a5d2-4c18-9fcf-3dbf5aa72ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:00.239438 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-18b104c7-a5d2-4c18-9fcf-3dbf5aa72ede None None] MFA Rules not processed for user `9c9149cb2b3c423796164b487a305bbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:20:00.240641 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-18b104c7-a5d2-4c18-9fcf-3dbf5aa72ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:20:00.288294 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:20:00.296863 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1212/2456] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:20:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:20:02.109727 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf75757f-38f0-4140-806e-d0f8cb737c5c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:02.111464 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf75757f-38f0-4140-806e-d0f8cb737c5c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:02.111965 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf75757f-38f0-4140-806e-d0f8cb737c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:02.118825 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=47 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:20:02.132386 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-cf75757f-38f0-4140-806e-d0f8cb737c5c None None] MFA Rules not processed for user `09c23d1c732c452a9597c647ad0241f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:20:02.133605 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cf75757f-38f0-4140-806e-d0f8cb737c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:20:02.140958 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1245/2457] 10.222.0.50 () {58 vars in 967 bytes} [Fri May 27 15:20:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:20:02.206019 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b104c7-a5d2-4c18-9fcf-3dbf5aa72ede None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:20:02.220064 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b104c7-a5d2-4c18-9fcf-3dbf5aa72ede None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:20:02.226299 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f49aff-d5d5-4dea-b670-d1dd9e171111 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18903f2eab7c415096e075d47f0b9a4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:20:02.226992 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f49aff-d5d5-4dea-b670-d1dd9e171111 service swift] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:02.227283 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f49aff-d5d5-4dea-b670-d1dd9e171111 service swift] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:02.227601 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f49aff-d5d5-4dea-b670-d1dd9e171111 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:02.237326 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f49aff-d5d5-4dea-b670-d1dd9e171111 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:20:02.238981 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f49aff-d5d5-4dea-b670-d1dd9e171111 service swift] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:20:02.248035 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1213/2458] 10.222.0.50 () {60 vars in 1384 bytes} [Fri May 27 15:20:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 27 15:20:07.547839 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf75757f-38f0-4140-806e-d0f8cb737c5c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:20:07.588941 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf75757f-38f0-4140-806e-d0f8cb737c5c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:20:07.609112 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32a339a-8e45-4edc-8579-207277918c35 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8293dab3ffd647f5927d323572fdbdce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:20:07.612829 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a339a-8e45-4edc-8579-207277918c35 service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:07.615539 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a339a-8e45-4edc-8579-207277918c35 service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:07.616304 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a339a-8e45-4edc-8579-207277918c35 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:07.627038 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f32a339a-8e45-4edc-8579-207277918c35 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:20:07.674371 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32a339a-8e45-4edc-8579-207277918c35 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:20:07.678436 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32a339a-8e45-4edc-8579-207277918c35 service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:20:07.708212 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1246/2459] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:20:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:20:12.244210 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=6 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:20:12.470989 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f49aff-d5d5-4dea-b670-d1dd9e171111 service swift] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:20:12.472657 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f9f49aff-d5d5-4dea-b670-d1dd9e171111 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:20:12.492904 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f49aff-d5d5-4dea-b670-d1dd9e171111 service swift] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:20:12.498423 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06744f6e-df0b-4191-92d2-9c640443f912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:20:12.499589 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06744f6e-df0b-4191-92d2-9c640443f912 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:12.500055 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06744f6e-df0b-4191-92d2-9c640443f912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:12.500602 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06744f6e-df0b-4191-92d2-9c640443f912 admin admin] PATH_INFO: `/v3/users/50ca9d699b004df4becfe3241802fb17` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:12.502898 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06744f6e-df0b-4191-92d2-9c640443f912 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50ca9d699b004df4becfe3241802fb17)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:20:12.505471 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06744f6e-df0b-4191-92d2-9c640443f912 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:20:12.581596 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-06744f6e-df0b-4191-92d2-9c640443f912 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50ca9d699b004df4becfe3241802fb17'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:12.596886 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-06744f6e-df0b-4191-92d2-9c640443f912 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50ca9d699b004df4becfe3241802fb17'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:12.606767 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-06744f6e-df0b-4191-92d2-9c640443f912 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50ca9d699b004df4becfe3241802fb17'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:12.608310 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-06744f6e-df0b-4191-92d2-9c640443f912 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50ca9d699b004df4becfe3241802fb17'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:12.609371 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-06744f6e-df0b-4191-92d2-9c640443f912 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50ca9d699b004df4becfe3241802fb17'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:12.640230 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1214/2460] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:20:12 2022] DELETE /identity/v3/users/50ca9d699b004df4becfe3241802fb17 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:20:13.337441 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32a339a-8e45-4edc-8579-207277918c35 service placement] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:20:13.362101 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32a339a-8e45-4edc-8579-207277918c35 service placement] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:20:13.468059 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1aaa08-74a9-4793-bcfa-e2d101ea064b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:20:13.469453 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1aaa08-74a9-4793-bcfa-e2d101ea064b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:13.473297 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1aaa08-74a9-4793-bcfa-e2d101ea064b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:13.473297 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1aaa08-74a9-4793-bcfa-e2d101ea064b admin admin] PATH_INFO: `/v3/projects/7facd29ad0d746099537a1b1345c2c90` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:13.473297 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1aaa08-74a9-4793-bcfa-e2d101ea064b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7facd29ad0d746099537a1b1345c2c90)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:20:13.474776 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1aaa08-74a9-4793-bcfa-e2d101ea064b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:20:13.549770 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bf1aaa08-74a9-4793-bcfa-e2d101ea064b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7facd29ad0d746099537a1b1345c2c90', 'request_id': 'req-bf1aaa08-74a9-4793-bcfa-e2d101ea064b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:13.575083 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bf1aaa08-74a9-4793-bcfa-e2d101ea064b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7facd29ad0d746099537a1b1345c2c90', 'request_id': 'req-bf1aaa08-74a9-4793-bcfa-e2d101ea064b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:13.590082 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bf1aaa08-74a9-4793-bcfa-e2d101ea064b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7facd29ad0d746099537a1b1345c2c90', 'request_id': 'req-bf1aaa08-74a9-4793-bcfa-e2d101ea064b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:13.621934 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bf1aaa08-74a9-4793-bcfa-e2d101ea064b admin admin] The token cache is being invalidate because project 7facd29ad0d746099537a1b1345c2c90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:20:13.622481 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bf1aaa08-74a9-4793-bcfa-e2d101ea064b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:13.630116 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bf1aaa08-74a9-4793-bcfa-e2d101ea064b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:13.632267 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1247/2461] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:20:13 2022] DELETE /identity/v3/projects/7facd29ad0d746099537a1b1345c2c90 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:20:13.637736 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06744f6e-df0b-4191-92d2-9c640443f912 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:20:13.639794 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-06744f6e-df0b-4191-92d2-9c640443f912 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:20:13.666888 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06744f6e-df0b-4191-92d2-9c640443f912 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:20:13.761119 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e83c3-e339-43e0-abeb-b13c66311e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:20:13.762825 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e83c3-e339-43e0-abeb-b13c66311e38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:13.763345 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e83c3-e339-43e0-abeb-b13c66311e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:13.763838 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794e83c3-e339-43e0-abeb-b13c66311e38 admin admin] PATH_INFO: `/v3/users/18ff5bfbc9f8453b981fb228ca2c1b6c` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:13.765623 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794e83c3-e339-43e0-abeb-b13c66311e38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18ff5bfbc9f8453b981fb228ca2c1b6c)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:20:13.768440 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794e83c3-e339-43e0-abeb-b13c66311e38 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:20:13.837316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-794e83c3-e339-43e0-abeb-b13c66311e38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18ff5bfbc9f8453b981fb228ca2c1b6c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:13.847316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-794e83c3-e339-43e0-abeb-b13c66311e38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18ff5bfbc9f8453b981fb228ca2c1b6c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:13.856313 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-794e83c3-e339-43e0-abeb-b13c66311e38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18ff5bfbc9f8453b981fb228ca2c1b6c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:13.857459 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-794e83c3-e339-43e0-abeb-b13c66311e38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18ff5bfbc9f8453b981fb228ca2c1b6c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:13.858331 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-794e83c3-e339-43e0-abeb-b13c66311e38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18ff5bfbc9f8453b981fb228ca2c1b6c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:13.871280 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:20:13.886113 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1215/2462] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:20:13 2022] DELETE /identity/v3/users/18ff5bfbc9f8453b981fb228ca2c1b6c => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:20:14.094204 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1aaa08-74a9-4793-bcfa-e2d101ea064b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:20:14.096963 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bf1aaa08-74a9-4793-bcfa-e2d101ea064b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:20:14.273155 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1aaa08-74a9-4793-bcfa-e2d101ea064b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:20:14.283625 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:20:14.373672 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab32b1c-a248-47e1-ad75-8e8e881107fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '451a9232f95e494683d81b6078a318d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:20:14.376440 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab32b1c-a248-47e1-ad75-8e8e881107fc service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:14.377125 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab32b1c-a248-47e1-ad75-8e8e881107fc service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:14.377684 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab32b1c-a248-47e1-ad75-8e8e881107fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:14.384048 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2ab32b1c-a248-47e1-ad75-8e8e881107fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:20:14.417936 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab32b1c-a248-47e1-ad75-8e8e881107fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:20:14.422022 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab32b1c-a248-47e1-ad75-8e8e881107fc service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:20:14.470475 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1248/2463] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:20:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:20:15.237186 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e83c3-e339-43e0-abeb-b13c66311e38 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:20:15.239537 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-794e83c3-e339-43e0-abeb-b13c66311e38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:20:15.354561 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794e83c3-e339-43e0-abeb-b13c66311e38 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:20:15.360000 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb16eb-d933-409f-a736-56686ab5013c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:20:15.361085 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb16eb-d933-409f-a736-56686ab5013c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:15.361372 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb16eb-d933-409f-a736-56686ab5013c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:15.361659 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecb16eb-d933-409f-a736-56686ab5013c admin admin] PATH_INFO: `/v3/projects/a4313b863b7f453587d5708db79e2143` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:15.363032 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb16eb-d933-409f-a736-56686ab5013c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4313b863b7f453587d5708db79e2143)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:20:15.364981 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecb16eb-d933-409f-a736-56686ab5013c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:20:15.448942 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0ecb16eb-d933-409f-a736-56686ab5013c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4313b863b7f453587d5708db79e2143', 'request_id': 'req-0ecb16eb-d933-409f-a736-56686ab5013c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:15.456295 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0ecb16eb-d933-409f-a736-56686ab5013c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4313b863b7f453587d5708db79e2143', 'request_id': 'req-0ecb16eb-d933-409f-a736-56686ab5013c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:15.463995 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0ecb16eb-d933-409f-a736-56686ab5013c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4313b863b7f453587d5708db79e2143', 'request_id': 'req-0ecb16eb-d933-409f-a736-56686ab5013c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:15.479770 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0ecb16eb-d933-409f-a736-56686ab5013c admin admin] The token cache is being invalidate because project a4313b863b7f453587d5708db79e2143 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:20:15.480318 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0ecb16eb-d933-409f-a736-56686ab5013c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:15.481178 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0ecb16eb-d933-409f-a736-56686ab5013c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:15.482898 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1216/2464] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:20:15 2022] DELETE /identity/v3/projects/a4313b863b7f453587d5708db79e2143 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:20:15.851024 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dcdac7-2f5b-4ca7-831c-5eaed6084ba4 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:15.852253 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dcdac7-2f5b-4ca7-831c-5eaed6084ba4 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:15.852675 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dcdac7-2f5b-4ca7-831c-5eaed6084ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:15.879149 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a2dcdac7-2f5b-4ca7-831c-5eaed6084ba4 None None] MFA Rules not processed for user `af74115f423c43fdb2544876be4e5622`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:20:15.880908 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a2dcdac7-2f5b-4ca7-831c-5eaed6084ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:20:15.965663 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1249/2465] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:20:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:20:17.875352 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb16eb-d933-409f-a736-56686ab5013c admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:20:17.877697 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0ecb16eb-d933-409f-a736-56686ab5013c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:20:18.031091 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecb16eb-d933-409f-a736-56686ab5013c admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:20:18.058812 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959ede59-d713-4a1e-9c66-09244c3f5160 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '451a9232f95e494683d81b6078a318d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:20:18.060070 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ede59-d713-4a1e-9c66-09244c3f5160 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:18.061873 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ede59-d713-4a1e-9c66-09244c3f5160 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:18.062452 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959ede59-d713-4a1e-9c66-09244c3f5160 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:18.066880 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-959ede59-d713-4a1e-9c66-09244c3f5160 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:20:18.096179 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959ede59-d713-4a1e-9c66-09244c3f5160 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:20:18.098427 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959ede59-d713-4a1e-9c66-09244c3f5160 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:20:18.117382 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1217/2466] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:20:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:20:20.940831 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dcdac7-2f5b-4ca7-831c-5eaed6084ba4 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:20:20.942954 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a2dcdac7-2f5b-4ca7-831c-5eaed6084ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:20:20.961628 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dcdac7-2f5b-4ca7-831c-5eaed6084ba4 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:20:20.968194 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf301f-a9b1-4ea5-90d2-ae95afb57aa2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '451a9232f95e494683d81b6078a318d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:20:20.970367 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf301f-a9b1-4ea5-90d2-ae95afb57aa2 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:20.970884 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf301f-a9b1-4ea5-90d2-ae95afb57aa2 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:20.971651 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf301f-a9b1-4ea5-90d2-ae95afb57aa2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:20.974779 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-56bf301f-a9b1-4ea5-90d2-ae95afb57aa2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:20:20.998218 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bf301f-a9b1-4ea5-90d2-ae95afb57aa2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:20:21.000274 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bf301f-a9b1-4ea5-90d2-ae95afb57aa2 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:20:21.013957 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1250/2467] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:20:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:20:28.109288 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=88,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:20:31.008977 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=60 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:20:46.073755 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959ede59-d713-4a1e-9c66-09244c3f5160 service nova] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:20:46.075314 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-959ede59-d713-4a1e-9c66-09244c3f5160 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:20:46.139440 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959ede59-d713-4a1e-9c66-09244c3f5160 service nova] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:20:46.145062 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d972457-7e30-4810-b2e4-b5e7ac8ce3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:20:46.145925 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d972457-7e30-4810-b2e4-b5e7ac8ce3b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:46.146206 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d972457-7e30-4810-b2e4-b5e7ac8ce3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:46.146480 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d972457-7e30-4810-b2e4-b5e7ac8ce3b0 admin admin] PATH_INFO: `/v3/users/0b257d6c8a034981be9b54789c313d3c` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:46.147708 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d972457-7e30-4810-b2e4-b5e7ac8ce3b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b257d6c8a034981be9b54789c313d3c)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:20:46.149642 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d972457-7e30-4810-b2e4-b5e7ac8ce3b0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:20:46.196623 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2d972457-7e30-4810-b2e4-b5e7ac8ce3b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b257d6c8a034981be9b54789c313d3c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:46.204242 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2d972457-7e30-4810-b2e4-b5e7ac8ce3b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b257d6c8a034981be9b54789c313d3c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:46.211285 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2d972457-7e30-4810-b2e4-b5e7ac8ce3b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b257d6c8a034981be9b54789c313d3c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:46.211943 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2d972457-7e30-4810-b2e4-b5e7ac8ce3b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b257d6c8a034981be9b54789c313d3c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:46.212555 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-2d972457-7e30-4810-b2e4-b5e7ac8ce3b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b257d6c8a034981be9b54789c313d3c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:46.229797 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1218/2468] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:20:46 2022] DELETE /identity/v3/users/0b257d6c8a034981be9b54789c313d3c => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:20:46.400806 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf301f-a9b1-4ea5-90d2-ae95afb57aa2 service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:20:46.417201 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf301f-a9b1-4ea5-90d2-ae95afb57aa2 service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:20:46.474367 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d2a351-a4bc-4633-af95-8a9534996618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:20:46.475305 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d2a351-a4bc-4633-af95-8a9534996618 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:46.475598 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d2a351-a4bc-4633-af95-8a9534996618 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:46.475874 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d2a351-a4bc-4633-af95-8a9534996618 admin admin] PATH_INFO: `/v3/projects/58318fdd40dd469099dade09793f6b38` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:46.477082 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d2a351-a4bc-4633-af95-8a9534996618 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58318fdd40dd469099dade09793f6b38)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:20:46.479143 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d2a351-a4bc-4633-af95-8a9534996618 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:20:46.525257 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-80d2a351-a4bc-4633-af95-8a9534996618 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58318fdd40dd469099dade09793f6b38', 'request_id': 'req-80d2a351-a4bc-4633-af95-8a9534996618'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:46.532434 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-80d2a351-a4bc-4633-af95-8a9534996618 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58318fdd40dd469099dade09793f6b38', 'request_id': 'req-80d2a351-a4bc-4633-af95-8a9534996618'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:46.538341 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-80d2a351-a4bc-4633-af95-8a9534996618 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58318fdd40dd469099dade09793f6b38', 'request_id': 'req-80d2a351-a4bc-4633-af95-8a9534996618'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:46.549232 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-80d2a351-a4bc-4633-af95-8a9534996618 admin admin] The token cache is being invalidate because project 58318fdd40dd469099dade09793f6b38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:20:46.549523 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-80d2a351-a4bc-4633-af95-8a9534996618 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:46.551123 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-80d2a351-a4bc-4633-af95-8a9534996618 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:20:46.552874 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1251/2469] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:20:46 2022] DELETE /identity/v3/projects/58318fdd40dd469099dade09793f6b38 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:20:55.730704 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d972457-7e30-4810-b2e4-b5e7ac8ce3b0 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:20:55.732167 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2d972457-7e30-4810-b2e4-b5e7ac8ce3b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:20:55.788280 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d972457-7e30-4810-b2e4-b5e7ac8ce3b0 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:20:55.819676 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:20:55.838588 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e964db0d-b88b-4303-be61-a4e3c943c885 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:20:55.839263 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e964db0d-b88b-4303-be61-a4e3c943c885 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:20:55.839550 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e964db0d-b88b-4303-be61-a4e3c943c885 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:20:55.839827 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e964db0d-b88b-4303-be61-a4e3c943c885 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:20:55.841715 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e964db0d-b88b-4303-be61-a4e3c943c885 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:20:55.898390 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e964db0d-b88b-4303-be61-a4e3c943c885 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:20:55.899844 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e964db0d-b88b-4303-be61-a4e3c943c885 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:20:55.908825 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1219/2470] 10.222.0.50 () {60 vars in 1396 bytes} [Fri May 27 15:20:55 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:20:56.546187 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:21:05.904542 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=32 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:21:13.022235 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d2a351-a4bc-4633-af95-8a9534996618 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:13.024337 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-80d2a351-a4bc-4633-af95-8a9534996618 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:13.036710 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d2a351-a4bc-4633-af95-8a9534996618 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:13.041143 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73a109d-b53f-47bf-b444-c42ad20b311d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:13.041869 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a109d-b53f-47bf-b444-c42ad20b311d service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:13.042168 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a109d-b53f-47bf-b444-c42ad20b311d service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:13.042470 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a109d-b53f-47bf-b444-c42ad20b311d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:13.044291 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c73a109d-b53f-47bf-b444-c42ad20b311d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:13.110956 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73a109d-b53f-47bf-b444-c42ad20b311d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:13.112408 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73a109d-b53f-47bf-b444-c42ad20b311d service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:13.122238 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1252/2471] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:21:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:21:16.364728 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e964db0d-b88b-4303-be61-a4e3c943c885 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:16.366366 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e964db0d-b88b-4303-be61-a4e3c943c885 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:16.380613 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e964db0d-b88b-4303-be61-a4e3c943c885 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:16.385759 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc0b4b-3f57-4fd8-8f09-aa234fc0c0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:16.386637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc0b4b-3f57-4fd8-8f09-aa234fc0c0dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:16.386990 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc0b4b-3f57-4fd8-8f09-aa234fc0c0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:16.387274 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc0b4b-3f57-4fd8-8f09-aa234fc0c0dd admin admin] PATH_INFO: `/v3/users/9c9149cb2b3c423796164b487a305bbd` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:16.388484 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dc0b4b-3f57-4fd8-8f09-aa234fc0c0dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c9149cb2b3c423796164b487a305bbd)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:16.390425 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dc0b4b-3f57-4fd8-8f09-aa234fc0c0dd admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:16.446512 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-45dc0b4b-3f57-4fd8-8f09-aa234fc0c0dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c9149cb2b3c423796164b487a305bbd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:16.455371 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-45dc0b4b-3f57-4fd8-8f09-aa234fc0c0dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c9149cb2b3c423796164b487a305bbd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:16.464192 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-45dc0b4b-3f57-4fd8-8f09-aa234fc0c0dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c9149cb2b3c423796164b487a305bbd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:16.464853 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-45dc0b4b-3f57-4fd8-8f09-aa234fc0c0dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c9149cb2b3c423796164b487a305bbd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:16.465458 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-45dc0b4b-3f57-4fd8-8f09-aa234fc0c0dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c9149cb2b3c423796164b487a305bbd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:16.483904 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1220/2472] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:21:16 2022] DELETE /identity/v3/users/9c9149cb2b3c423796164b487a305bbd => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:21:16.661174 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73a109d-b53f-47bf-b444-c42ad20b311d service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:16.671457 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73a109d-b53f-47bf-b444-c42ad20b311d service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:16.730647 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e642da38-cef8-4065-9105-f77557689d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:16.731542 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e642da38-cef8-4065-9105-f77557689d17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:16.731833 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e642da38-cef8-4065-9105-f77557689d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:16.732111 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e642da38-cef8-4065-9105-f77557689d17 admin admin] PATH_INFO: `/v3/projects/c208c3d86c504ea086bf763e241cbc88` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:16.733274 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e642da38-cef8-4065-9105-f77557689d17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c208c3d86c504ea086bf763e241cbc88)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:16.735199 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e642da38-cef8-4065-9105-f77557689d17 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:16.780240 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e642da38-cef8-4065-9105-f77557689d17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c208c3d86c504ea086bf763e241cbc88', 'request_id': 'req-e642da38-cef8-4065-9105-f77557689d17'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:16.787355 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e642da38-cef8-4065-9105-f77557689d17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c208c3d86c504ea086bf763e241cbc88', 'request_id': 'req-e642da38-cef8-4065-9105-f77557689d17'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:16.793042 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e642da38-cef8-4065-9105-f77557689d17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c208c3d86c504ea086bf763e241cbc88', 'request_id': 'req-e642da38-cef8-4065-9105-f77557689d17'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:16.804838 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e642da38-cef8-4065-9105-f77557689d17 admin admin] The token cache is being invalidate because project c208c3d86c504ea086bf763e241cbc88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:21:16.805132 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e642da38-cef8-4065-9105-f77557689d17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:16.805764 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e642da38-cef8-4065-9105-f77557689d17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:16.807364 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1253/2473] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:21:16 2022] DELETE /identity/v3/projects/c208c3d86c504ea086bf763e241cbc88 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:21:16.812273 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f0bab5-a6a3-4ead-8e87-1e059c46a1b1 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:16.812566 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f0bab5-a6a3-4ead-8e87-1e059c46a1b1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:16.812847 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f0bab5-a6a3-4ead-8e87-1e059c46a1b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:16.831951 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-32f0bab5-a6a3-4ead-8e87-1e059c46a1b1 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:21:16.833131 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-32f0bab5-a6a3-4ead-8e87-1e059c46a1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:16.893973 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1221/2474] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:21:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:21:16.901749 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ebd66b-ff17-477f-824b-3143108fdeee None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:16.902046 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ebd66b-ff17-477f-824b-3143108fdeee None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:16.902329 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ebd66b-ff17-477f-824b-3143108fdeee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:16.920328 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-40ebd66b-ff17-477f-824b-3143108fdeee None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:21:16.921532 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-40ebd66b-ff17-477f-824b-3143108fdeee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:16.930343 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1254/2475] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:21:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:21:16.934372 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f0bab5-a6a3-4ead-8e87-1e059c46a1b1 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:16.943910 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f0bab5-a6a3-4ead-8e87-1e059c46a1b1 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:16.949357 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3ac89-2a70-4fcd-9117-7cfcc3446a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:16.950018 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3ac89-2a70-4fcd-9117-7cfcc3446a2d admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:16.950305 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3ac89-2a70-4fcd-9117-7cfcc3446a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:16.950609 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3ac89-2a70-4fcd-9117-7cfcc3446a2d admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:16.951376 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a3ac89-2a70-4fcd-9117-7cfcc3446a2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:21:16.951692 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a3ac89-2a70-4fcd-9117-7cfcc3446a2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:16.953308 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a3ac89-2a70-4fcd-9117-7cfcc3446a2d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:16.968665 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1222/2476] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:21:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:21:16.970038 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ebd66b-ff17-477f-824b-3143108fdeee None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:16.981046 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ebd66b-ff17-477f-824b-3143108fdeee None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:16.985958 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee3f32-fce9-4b18-a016-5e60758ccba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:16.986581 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee3f32-fce9-4b18-a016-5e60758ccba9 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:16.986906 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee3f32-fce9-4b18-a016-5e60758ccba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:16.987191 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ee3f32-fce9-4b18-a016-5e60758ccba9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:16.987880 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ee3f32-fce9-4b18-a016-5e60758ccba9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:16.989765 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ee3f32-fce9-4b18-a016-5e60758ccba9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:17.004568 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:21:17.031356 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1255/2477] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:21:16 2022] POST /identity/v3/projects => generated 369 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:21:17.032460 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3ac89-2a70-4fcd-9117-7cfcc3446a2d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:17.042598 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3ac89-2a70-4fcd-9117-7cfcc3446a2d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:17.102662 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01bb14f-51f7-4098-94be-50f7bac74657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:17.103280 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01bb14f-51f7-4098-94be-50f7bac74657 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:17.103574 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01bb14f-51f7-4098-94be-50f7bac74657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:17.103836 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01bb14f-51f7-4098-94be-50f7bac74657 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:17.104528 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01bb14f-51f7-4098-94be-50f7bac74657 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:17.106447 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01bb14f-51f7-4098-94be-50f7bac74657 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:17.113228 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:21:17.131198 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1223/2478] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:21:17 2022] POST /identity/v3/users => generated 285 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:21:17.133064 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee3f32-fce9-4b18-a016-5e60758ccba9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:17.143285 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ee3f32-fce9-4b18-a016-5e60758ccba9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:17.147932 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df19c54-189e-41b1-b2f3-0a8a1cff6af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:17.149078 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df19c54-189e-41b1-b2f3-0a8a1cff6af9 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:17.149634 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df19c54-189e-41b1-b2f3-0a8a1cff6af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:17.150141 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df19c54-189e-41b1-b2f3-0a8a1cff6af9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:17.150927 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df19c54-189e-41b1-b2f3-0a8a1cff6af9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:17.152523 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df19c54-189e-41b1-b2f3-0a8a1cff6af9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:17.162870 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1256/2479] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:21:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:21:17.168023 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01bb14f-51f7-4098-94be-50f7bac74657 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:17.178034 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01bb14f-51f7-4098-94be-50f7bac74657 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:17.183554 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4c5df1-1653-4f3a-9535-40ef1d4662bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:17.184206 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c5df1-1653-4f3a-9535-40ef1d4662bd admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:17.184485 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c5df1-1653-4f3a-9535-40ef1d4662bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:17.184754 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4c5df1-1653-4f3a-9535-40ef1d4662bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:17.185458 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4c5df1-1653-4f3a-9535-40ef1d4662bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:17.187080 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4c5df1-1653-4f3a-9535-40ef1d4662bd admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:17.197227 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1224/2480] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:21:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:21:17.201546 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df19c54-189e-41b1-b2f3-0a8a1cff6af9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:17.211668 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df19c54-189e-41b1-b2f3-0a8a1cff6af9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:17.216167 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a3e761-3eef-49c9-83d7-bf6c7e8970e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:17.217033 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a3e761-3eef-49c9-83d7-bf6c7e8970e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:17.217315 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a3e761-3eef-49c9-83d7-bf6c7e8970e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:17.217604 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a3e761-3eef-49c9-83d7-bf6c7e8970e9 admin admin] PATH_INFO: `/v3/projects/160e10e7f7b54e0bb74f4fe2ccfa9c92/users/5b3a346eae7d4acaa65784c2a54c43b7/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:17.233288 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a3e761-3eef-49c9-83d7-bf6c7e8970e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=160e10e7f7b54e0bb74f4fe2ccfa9c92, user_id=5b3a346eae7d4acaa65784c2a54c43b7, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:17.236126 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a3e761-3eef-49c9-83d7-bf6c7e8970e9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:17.252241 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1257/2481] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:21:17 2022] PUT /identity/v3/projects/160e10e7f7b54e0bb74f4fe2ccfa9c92/users/5b3a346eae7d4acaa65784c2a54c43b7/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:21:17.263228 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076a791-b11d-45b6-99ab-6dec9e4a06b2 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:17.263527 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076a791-b11d-45b6-99ab-6dec9e4a06b2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:17.263809 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076a791-b11d-45b6-99ab-6dec9e4a06b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:17.290703 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-2076a791-b11d-45b6-99ab-6dec9e4a06b2 None None] MFA Rules not processed for user `5b3a346eae7d4acaa65784c2a54c43b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:21:17.291966 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-2076a791-b11d-45b6-99ab-6dec9e4a06b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:17.357540 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1225/2482] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:21:17 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:21:17.362436 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a3e761-3eef-49c9-83d7-bf6c7e8970e9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:17.427846 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a3e761-3eef-49c9-83d7-bf6c7e8970e9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:17.433199 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0928aac-bbbf-4844-b6be-061702244e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:17.433855 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0928aac-bbbf-4844-b6be-061702244e23 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:17.434141 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0928aac-bbbf-4844-b6be-061702244e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:17.434423 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0928aac-bbbf-4844-b6be-061702244e23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:17.435147 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0928aac-bbbf-4844-b6be-061702244e23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:17.437056 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0928aac-bbbf-4844-b6be-061702244e23 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:17.462097 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1258/2483] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:21:17 2022] POST /identity/v3/projects => generated 367 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:21:17.466153 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2076a791-b11d-45b6-99ab-6dec9e4a06b2 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:17.477490 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2076a791-b11d-45b6-99ab-6dec9e4a06b2 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:17.531836 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc1889b-ab0f-48d5-a337-4be7525a8969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:17.532504 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc1889b-ab0f-48d5-a337-4be7525a8969 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:17.532789 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc1889b-ab0f-48d5-a337-4be7525a8969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:17.533065 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc1889b-ab0f-48d5-a337-4be7525a8969 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:17.533760 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc1889b-ab0f-48d5-a337-4be7525a8969 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:17.535706 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc1889b-ab0f-48d5-a337-4be7525a8969 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:17.555562 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1226/2484] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:21:17 2022] POST /identity/v3/users => generated 290 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:21:17.559483 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0928aac-bbbf-4844-b6be-061702244e23 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:17.569708 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0928aac-bbbf-4844-b6be-061702244e23 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:17.574287 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d38eeac-aaa9-43c3-b24a-e8a2c2fe69c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:17.574982 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d38eeac-aaa9-43c3-b24a-e8a2c2fe69c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:17.575260 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d38eeac-aaa9-43c3-b24a-e8a2c2fe69c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:17.575533 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d38eeac-aaa9-43c3-b24a-e8a2c2fe69c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:17.576254 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d38eeac-aaa9-43c3-b24a-e8a2c2fe69c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:17.577834 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d38eeac-aaa9-43c3-b24a-e8a2c2fe69c8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:17.588888 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1259/2485] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:21:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:21:17.593152 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc1889b-ab0f-48d5-a337-4be7525a8969 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:17.603922 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc1889b-ab0f-48d5-a337-4be7525a8969 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:17.608602 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3074b9-3414-4752-b84a-300e33a96aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:17.609419 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3074b9-3414-4752-b84a-300e33a96aeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:17.609706 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3074b9-3414-4752-b84a-300e33a96aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:17.609965 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3074b9-3414-4752-b84a-300e33a96aeb admin admin] PATH_INFO: `/v3/projects/a0ff4fe0ded248f78330d2ca6db75d41/users/5164451ec9964b90af6da73d695f8bca/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:17.624697 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3074b9-3414-4752-b84a-300e33a96aeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0ff4fe0ded248f78330d2ca6db75d41, user_id=5164451ec9964b90af6da73d695f8bca, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:17.627402 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3074b9-3414-4752-b84a-300e33a96aeb admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:17.640470 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1227/2486] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:21:17 2022] PUT /identity/v3/projects/a0ff4fe0ded248f78330d2ca6db75d41/users/5164451ec9964b90af6da73d695f8bca/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:21:17.644402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d38eeac-aaa9-43c3-b24a-e8a2c2fe69c8 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:17.655070 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d38eeac-aaa9-43c3-b24a-e8a2c2fe69c8 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:17.710002 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3964591f-9a25-4bee-ada8-15d34d56c030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:17.711471 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964591f-9a25-4bee-ada8-15d34d56c030 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:17.711768 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964591f-9a25-4bee-ada8-15d34d56c030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:17.712045 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3964591f-9a25-4bee-ada8-15d34d56c030 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:17.712767 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3964591f-9a25-4bee-ada8-15d34d56c030 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:17.714378 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3964591f-9a25-4bee-ada8-15d34d56c030 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:17.726997 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1260/2487] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:21:17 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:21:17.729689 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3074b9-3414-4752-b84a-300e33a96aeb admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:17.740048 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3074b9-3414-4752-b84a-300e33a96aeb admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:17.745014 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac527c53-e4e5-4aaf-819c-198bb58197bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:17.745895 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac527c53-e4e5-4aaf-819c-198bb58197bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:17.746177 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac527c53-e4e5-4aaf-819c-198bb58197bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:17.746453 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac527c53-e4e5-4aaf-819c-198bb58197bf admin admin] PATH_INFO: `/v3/projects/a0ff4fe0ded248f78330d2ca6db75d41/users/5164451ec9964b90af6da73d695f8bca/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:17.748679 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac527c53-e4e5-4aaf-819c-198bb58197bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0ff4fe0ded248f78330d2ca6db75d41, user_id=5164451ec9964b90af6da73d695f8bca, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:17.751437 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac527c53-e4e5-4aaf-819c-198bb58197bf admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:17.762572 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1228/2488] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:21:17 2022] PUT /identity/v3/projects/a0ff4fe0ded248f78330d2ca6db75d41/users/5164451ec9964b90af6da73d695f8bca/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:21:17.773851 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8db8b-568b-467d-8515-f9f5eed5f703 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:17.774139 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8db8b-568b-467d-8515-f9f5eed5f703 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:17.774433 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f8db8b-568b-467d-8515-f9f5eed5f703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:17.808500 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c1f8db8b-568b-467d-8515-f9f5eed5f703 None None] MFA Rules not processed for user `5164451ec9964b90af6da73d695f8bca`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:21:17.809752 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c1f8db8b-568b-467d-8515-f9f5eed5f703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:17.819068 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:21:17.880818 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1261/2489] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:21:17 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:21:17.891952 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac527c53-e4e5-4aaf-819c-198bb58197bf admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:17.893701 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ac527c53-e4e5-4aaf-819c-198bb58197bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:17.917904 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:21:17.959255 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac527c53-e4e5-4aaf-819c-198bb58197bf admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:17.963853 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246c76ef-dd7f-478e-b825-45ca05bd7762 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:17.964580 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c76ef-dd7f-478e-b825-45ca05bd7762 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:17.964870 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c76ef-dd7f-478e-b825-45ca05bd7762 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:17.965153 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246c76ef-dd7f-478e-b825-45ca05bd7762 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:17.975032 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246c76ef-dd7f-478e-b825-45ca05bd7762 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:17.976479 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246c76ef-dd7f-478e-b825-45ca05bd7762 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:17.990293 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1229/2490] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:21:17 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:21:19.381062 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f8db8b-568b-467d-8515-f9f5eed5f703 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:19.449796 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f8db8b-568b-467d-8515-f9f5eed5f703 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:19.455347 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a0d5a-159f-4f77-b1ea-8986b545d6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:19.456952 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a0d5a-159f-4f77-b1ea-8986b545d6b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:19.457240 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a0d5a-159f-4f77-b1ea-8986b545d6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:19.457518 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a0d5a-159f-4f77-b1ea-8986b545d6b2 admin admin] PATH_INFO: `/v3/users/5b3a346eae7d4acaa65784c2a54c43b7` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:19.458764 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457a0d5a-159f-4f77-b1ea-8986b545d6b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b3a346eae7d4acaa65784c2a54c43b7)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:19.465686 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457a0d5a-159f-4f77-b1ea-8986b545d6b2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:19.516938 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-457a0d5a-159f-4f77-b1ea-8986b545d6b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b3a346eae7d4acaa65784c2a54c43b7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:19.524856 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-457a0d5a-159f-4f77-b1ea-8986b545d6b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b3a346eae7d4acaa65784c2a54c43b7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:19.525534 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-457a0d5a-159f-4f77-b1ea-8986b545d6b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b3a346eae7d4acaa65784c2a54c43b7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:19.532793 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-457a0d5a-159f-4f77-b1ea-8986b545d6b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b3a346eae7d4acaa65784c2a54c43b7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:19.533406 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-457a0d5a-159f-4f77-b1ea-8986b545d6b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b3a346eae7d4acaa65784c2a54c43b7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:19.550328 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1262/2491] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:21:19 2022] DELETE /identity/v3/users/5b3a346eae7d4acaa65784c2a54c43b7 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:21:19.554871 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7400b41-9783-4d12-8321-44d8c1f56684 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:19.555351 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7400b41-9783-4d12-8321-44d8c1f56684 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:19.555448 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7400b41-9783-4d12-8321-44d8c1f56684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:19.573171 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f7400b41-9783-4d12-8321-44d8c1f56684 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:21:19.574344 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f7400b41-9783-4d12-8321-44d8c1f56684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:19.633593 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1230/2492] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:21:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:21:19.643150 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a0d5a-159f-4f77-b1ea-8986b545d6b2 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:19.644576 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-457a0d5a-159f-4f77-b1ea-8986b545d6b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:19.675000 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:21:19.702379 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a0d5a-159f-4f77-b1ea-8986b545d6b2 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:19.705991 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7696f3e-1aac-4bc1-ad32-639d8d29dde1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:19.706635 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7696f3e-1aac-4bc1-ad32-639d8d29dde1 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:19.706956 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7696f3e-1aac-4bc1-ad32-639d8d29dde1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:19.707241 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7696f3e-1aac-4bc1-ad32-639d8d29dde1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:19.716904 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7696f3e-1aac-4bc1-ad32-639d8d29dde1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:19.718335 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7696f3e-1aac-4bc1-ad32-639d8d29dde1 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:19.727953 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1263/2493] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:21:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:21:20.010126 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7400b41-9783-4d12-8321-44d8c1f56684 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:20.011612 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f7400b41-9783-4d12-8321-44d8c1f56684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:20.023461 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7400b41-9783-4d12-8321-44d8c1f56684 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:20.028275 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fe6fb6-9b8d-469f-a71d-f7e548d00ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:20.029126 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe6fb6-9b8d-469f-a71d-f7e548d00ea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:20.029405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe6fb6-9b8d-469f-a71d-f7e548d00ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:20.029977 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fe6fb6-9b8d-469f-a71d-f7e548d00ea6 admin admin] PATH_INFO: `/v3/projects/160e10e7f7b54e0bb74f4fe2ccfa9c92` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:20.030834 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fe6fb6-9b8d-469f-a71d-f7e548d00ea6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=160e10e7f7b54e0bb74f4fe2ccfa9c92)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:20.032766 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fe6fb6-9b8d-469f-a71d-f7e548d00ea6 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:20.080744 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-87fe6fb6-9b8d-469f-a71d-f7e548d00ea6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '160e10e7f7b54e0bb74f4fe2ccfa9c92', 'request_id': 'req-87fe6fb6-9b8d-469f-a71d-f7e548d00ea6'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:20.086622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-87fe6fb6-9b8d-469f-a71d-f7e548d00ea6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '160e10e7f7b54e0bb74f4fe2ccfa9c92', 'request_id': 'req-87fe6fb6-9b8d-469f-a71d-f7e548d00ea6'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:20.093414 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-87fe6fb6-9b8d-469f-a71d-f7e548d00ea6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '160e10e7f7b54e0bb74f4fe2ccfa9c92', 'request_id': 'req-87fe6fb6-9b8d-469f-a71d-f7e548d00ea6'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:20.106724 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-87fe6fb6-9b8d-469f-a71d-f7e548d00ea6 admin admin] The token cache is being invalidate because project 160e10e7f7b54e0bb74f4fe2ccfa9c92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:21:20.107061 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-87fe6fb6-9b8d-469f-a71d-f7e548d00ea6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:20.107709 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-87fe6fb6-9b8d-469f-a71d-f7e548d00ea6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:20.109326 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1231/2494] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:21:20 2022] DELETE /identity/v3/projects/160e10e7f7b54e0bb74f4fe2ccfa9c92 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:21:20.113499 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7696f3e-1aac-4bc1-ad32-639d8d29dde1 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:20.124904 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7696f3e-1aac-4bc1-ad32-639d8d29dde1 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:20.188440 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70397191-ae35-44e1-9874-67ad461a0737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:20.189627 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70397191-ae35-44e1-9874-67ad461a0737 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:20.189909 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70397191-ae35-44e1-9874-67ad461a0737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:20.190189 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70397191-ae35-44e1-9874-67ad461a0737 admin admin] PATH_INFO: `/v3/users/5164451ec9964b90af6da73d695f8bca` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:20.191473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70397191-ae35-44e1-9874-67ad461a0737 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5164451ec9964b90af6da73d695f8bca)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:20.193421 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70397191-ae35-44e1-9874-67ad461a0737 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:20.246660 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-70397191-ae35-44e1-9874-67ad461a0737 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5164451ec9964b90af6da73d695f8bca'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:20.255339 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-70397191-ae35-44e1-9874-67ad461a0737 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5164451ec9964b90af6da73d695f8bca'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:20.256073 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-70397191-ae35-44e1-9874-67ad461a0737 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5164451ec9964b90af6da73d695f8bca'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:20.266940 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-70397191-ae35-44e1-9874-67ad461a0737 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5164451ec9964b90af6da73d695f8bca'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:20.267751 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-70397191-ae35-44e1-9874-67ad461a0737 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5164451ec9964b90af6da73d695f8bca'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:20.289433 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1264/2495] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:21:20 2022] DELETE /identity/v3/users/5164451ec9964b90af6da73d695f8bca => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:21:20.578102 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fe6fb6-9b8d-469f-a71d-f7e548d00ea6 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:20.579544 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-87fe6fb6-9b8d-469f-a71d-f7e548d00ea6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:20.595206 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fe6fb6-9b8d-469f-a71d-f7e548d00ea6 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:20.640932 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:21:20.655819 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f93b83-7cd3-4e60-a282-b149e137bb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:20.656814 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f93b83-7cd3-4e60-a282-b149e137bb0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:20.657108 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f93b83-7cd3-4e60-a282-b149e137bb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:20.657460 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f93b83-7cd3-4e60-a282-b149e137bb0a admin admin] PATH_INFO: `/v3/projects/a0ff4fe0ded248f78330d2ca6db75d41` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:20.658605 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f93b83-7cd3-4e60-a282-b149e137bb0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0ff4fe0ded248f78330d2ca6db75d41)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:20.660597 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f93b83-7cd3-4e60-a282-b149e137bb0a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:20.707803 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e1f93b83-7cd3-4e60-a282-b149e137bb0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0ff4fe0ded248f78330d2ca6db75d41', 'request_id': 'req-e1f93b83-7cd3-4e60-a282-b149e137bb0a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:20.714235 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e1f93b83-7cd3-4e60-a282-b149e137bb0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0ff4fe0ded248f78330d2ca6db75d41', 'request_id': 'req-e1f93b83-7cd3-4e60-a282-b149e137bb0a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:20.722078 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e1f93b83-7cd3-4e60-a282-b149e137bb0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0ff4fe0ded248f78330d2ca6db75d41', 'request_id': 'req-e1f93b83-7cd3-4e60-a282-b149e137bb0a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:20.737647 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e1f93b83-7cd3-4e60-a282-b149e137bb0a admin admin] The token cache is being invalidate because project a0ff4fe0ded248f78330d2ca6db75d41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:21:20.737647 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e1f93b83-7cd3-4e60-a282-b149e137bb0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:20.738360 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e1f93b83-7cd3-4e60-a282-b149e137bb0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:20.743856 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1232/2496] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:21:20 2022] DELETE /identity/v3/projects/a0ff4fe0ded248f78330d2ca6db75d41 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:21:20.745411 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c87ce37-645f-4d93-97bf-4e76c690d594 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:20.745857 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c87ce37-645f-4d93-97bf-4e76c690d594 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:20.746027 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c87ce37-645f-4d93-97bf-4e76c690d594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:20.765316 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7c87ce37-645f-4d93-97bf-4e76c690d594 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:21:20.766517 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7c87ce37-645f-4d93-97bf-4e76c690d594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:20.822113 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:21:20.838966 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1265/2497] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:21:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:21:20.846747 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9098504-ec89-4eb9-91d9-c09924c39881 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:20.847072 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9098504-ec89-4eb9-91d9-c09924c39881 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:20.847353 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9098504-ec89-4eb9-91d9-c09924c39881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:20.867669 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f9098504-ec89-4eb9-91d9-c09924c39881 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:21:20.869312 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f9098504-ec89-4eb9-91d9-c09924c39881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:20.877220 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1233/2498] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:21:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:21:20.881534 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c87ce37-645f-4d93-97bf-4e76c690d594 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:20.892082 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c87ce37-645f-4d93-97bf-4e76c690d594 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:20.896767 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a4d7b2-4184-4a6b-b66b-fbda2514d29b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:20.897435 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a4d7b2-4184-4a6b-b66b-fbda2514d29b admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:20.898289 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a4d7b2-4184-4a6b-b66b-fbda2514d29b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:20.898605 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a4d7b2-4184-4a6b-b66b-fbda2514d29b admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:20.899474 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a4d7b2-4184-4a6b-b66b-fbda2514d29b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:21:20.899784 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a4d7b2-4184-4a6b-b66b-fbda2514d29b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:20.901432 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a4d7b2-4184-4a6b-b66b-fbda2514d29b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:20.919948 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1266/2499] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:21:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:21:20.924214 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9098504-ec89-4eb9-91d9-c09924c39881 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:20.933708 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9098504-ec89-4eb9-91d9-c09924c39881 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:20.938326 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e2d451-12a8-46be-939d-645ea63c3758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:20.939005 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2d451-12a8-46be-939d-645ea63c3758 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:20.939290 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2d451-12a8-46be-939d-645ea63c3758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:20.939567 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2d451-12a8-46be-939d-645ea63c3758 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:20.940260 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e2d451-12a8-46be-939d-645ea63c3758 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:20.942299 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e2d451-12a8-46be-939d-645ea63c3758 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:20.968803 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1234/2500] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:21:20 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:21:20.973132 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a4d7b2-4184-4a6b-b66b-fbda2514d29b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:20.983605 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a4d7b2-4184-4a6b-b66b-fbda2514d29b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:21.041686 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe439426-4f0e-479b-a3ae-dd943f4f6e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:21.042349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe439426-4f0e-479b-a3ae-dd943f4f6e6b admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:21.042628 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe439426-4f0e-479b-a3ae-dd943f4f6e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:21.042940 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe439426-4f0e-479b-a3ae-dd943f4f6e6b admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:21.043633 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe439426-4f0e-479b-a3ae-dd943f4f6e6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:21.045526 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe439426-4f0e-479b-a3ae-dd943f4f6e6b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:21.065056 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1267/2501] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:21:20 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:21:21.069170 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e2d451-12a8-46be-939d-645ea63c3758 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:21.079686 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e2d451-12a8-46be-939d-645ea63c3758 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:21.084166 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca42d1c-d030-49ce-92a0-cb175c456831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:21.084807 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca42d1c-d030-49ce-92a0-cb175c456831 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:21.085087 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca42d1c-d030-49ce-92a0-cb175c456831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:21.085907 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca42d1c-d030-49ce-92a0-cb175c456831 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:21.086621 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca42d1c-d030-49ce-92a0-cb175c456831 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:21.088242 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca42d1c-d030-49ce-92a0-cb175c456831 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:21.098658 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1235/2502] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:21:21.102652 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe439426-4f0e-479b-a3ae-dd943f4f6e6b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:21.113310 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe439426-4f0e-479b-a3ae-dd943f4f6e6b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:21.118698 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d479f6a-bc4c-4f56-a15a-89c04898cb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:21.119386 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d479f6a-bc4c-4f56-a15a-89c04898cb14 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:21.119668 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d479f6a-bc4c-4f56-a15a-89c04898cb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:21.119941 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d479f6a-bc4c-4f56-a15a-89c04898cb14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:21.120780 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d479f6a-bc4c-4f56-a15a-89c04898cb14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:21.123375 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d479f6a-bc4c-4f56-a15a-89c04898cb14 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:21.139988 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1268/2503] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:21:21.144285 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca42d1c-d030-49ce-92a0-cb175c456831 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:21.156637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca42d1c-d030-49ce-92a0-cb175c456831 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:21.162136 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401dc4e3-8bed-4dd8-825a-2840efa0937a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:21.163044 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401dc4e3-8bed-4dd8-825a-2840efa0937a admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:21.163371 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401dc4e3-8bed-4dd8-825a-2840efa0937a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:21.163646 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401dc4e3-8bed-4dd8-825a-2840efa0937a admin admin] PATH_INFO: `/v3/projects/07142202f39444fa8c3994886a292a91/users/8089293170fa43139ce2c3ad3591ba9a/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:21.178588 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401dc4e3-8bed-4dd8-825a-2840efa0937a admin admin] RBAC: Authorizing `identity:create_grant(project_id=07142202f39444fa8c3994886a292a91, user_id=8089293170fa43139ce2c3ad3591ba9a, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:21.181281 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401dc4e3-8bed-4dd8-825a-2840efa0937a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:21.193855 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1236/2504] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:21:21 2022] PUT /identity/v3/projects/07142202f39444fa8c3994886a292a91/users/8089293170fa43139ce2c3ad3591ba9a/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:21:21.205607 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea4dce-fc93-4d57-8148-96021ff376f1 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:21.205898 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea4dce-fc93-4d57-8148-96021ff376f1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:21.206182 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eea4dce-fc93-4d57-8148-96021ff376f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:21.232972 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2eea4dce-fc93-4d57-8148-96021ff376f1 None None] MFA Rules not processed for user `8089293170fa43139ce2c3ad3591ba9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:21:21.234169 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2eea4dce-fc93-4d57-8148-96021ff376f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:21.298188 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1269/2505] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:21:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:21:21.302632 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401dc4e3-8bed-4dd8-825a-2840efa0937a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:21.365891 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401dc4e3-8bed-4dd8-825a-2840efa0937a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:21.371133 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a720a-07e1-41ab-adc0-805ff99cbf5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:21.371790 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a720a-07e1-41ab-adc0-805ff99cbf5a admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:21.372074 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a720a-07e1-41ab-adc0-805ff99cbf5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:21.372350 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a720a-07e1-41ab-adc0-805ff99cbf5a admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:21.373031 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7a720a-07e1-41ab-adc0-805ff99cbf5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:21.374958 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7a720a-07e1-41ab-adc0-805ff99cbf5a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:21.393564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:21:21.402075 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1237/2506] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:21:21 2022] POST /identity/v3/projects => generated 387 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:21:21.406237 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eea4dce-fc93-4d57-8148-96021ff376f1 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:21.416977 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eea4dce-fc93-4d57-8148-96021ff376f1 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:21.457655 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:21:21.473116 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84537e42-4d2c-47f8-b962-af5dceffedd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:21.473796 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84537e42-4d2c-47f8-b962-af5dceffedd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:21.474091 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84537e42-4d2c-47f8-b962-af5dceffedd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:21.474367 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84537e42-4d2c-47f8-b962-af5dceffedd6 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:21.475080 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84537e42-4d2c-47f8-b962-af5dceffedd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:21.477934 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84537e42-4d2c-47f8-b962-af5dceffedd6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:21.498323 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1270/2507] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:21:21 2022] POST /identity/v3/users => generated 300 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:21:21.502664 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a720a-07e1-41ab-adc0-805ff99cbf5a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:21.513581 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a720a-07e1-41ab-adc0-805ff99cbf5a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:21.518145 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e135dd-3b71-4d82-9a76-925ca0d33f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:21.518806 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e135dd-3b71-4d82-9a76-925ca0d33f89 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:21.519119 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e135dd-3b71-4d82-9a76-925ca0d33f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:21.519398 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e135dd-3b71-4d82-9a76-925ca0d33f89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:21.520113 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e135dd-3b71-4d82-9a76-925ca0d33f89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:21.521733 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e135dd-3b71-4d82-9a76-925ca0d33f89 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:21.532440 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1238/2508] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:21:21.536487 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84537e42-4d2c-47f8-b962-af5dceffedd6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:21.547295 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84537e42-4d2c-47f8-b962-af5dceffedd6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:21.552281 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3607030-7ada-455b-9d63-0921e4e29330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:21.553086 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3607030-7ada-455b-9d63-0921e4e29330 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:21.553366 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3607030-7ada-455b-9d63-0921e4e29330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:21.553639 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3607030-7ada-455b-9d63-0921e4e29330 admin admin] PATH_INFO: `/v3/projects/26c30a98ebc84c6fb9d4952e10a6ef4b/users/e6277cc8ff4841a5a06dbad991b30923/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:21.569007 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3607030-7ada-455b-9d63-0921e4e29330 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26c30a98ebc84c6fb9d4952e10a6ef4b, user_id=e6277cc8ff4841a5a06dbad991b30923, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:21.572303 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3607030-7ada-455b-9d63-0921e4e29330 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:21.585902 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1271/2509] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:21:21 2022] PUT /identity/v3/projects/26c30a98ebc84c6fb9d4952e10a6ef4b/users/e6277cc8ff4841a5a06dbad991b30923/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:21:21.590212 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e135dd-3b71-4d82-9a76-925ca0d33f89 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:21.601606 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e135dd-3b71-4d82-9a76-925ca0d33f89 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:21.660885 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cae01c-115d-48ac-806b-6614566869e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:21.661555 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cae01c-115d-48ac-806b-6614566869e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:21.661840 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cae01c-115d-48ac-806b-6614566869e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:21.662118 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cae01c-115d-48ac-806b-6614566869e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:21.662863 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cae01c-115d-48ac-806b-6614566869e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:21.665089 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cae01c-115d-48ac-806b-6614566869e3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:21.675140 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1239/2510] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:21:21.679888 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3607030-7ada-455b-9d63-0921e4e29330 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:21.690230 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3607030-7ada-455b-9d63-0921e4e29330 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:21.694902 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeea315-05c8-406d-a145-e41074e6a0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:21.695838 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeea315-05c8-406d-a145-e41074e6a0ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:21.696193 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeea315-05c8-406d-a145-e41074e6a0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:21.696549 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeea315-05c8-406d-a145-e41074e6a0ce admin admin] PATH_INFO: `/v3/projects/26c30a98ebc84c6fb9d4952e10a6ef4b/users/e6277cc8ff4841a5a06dbad991b30923/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:21.698568 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeea315-05c8-406d-a145-e41074e6a0ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=26c30a98ebc84c6fb9d4952e10a6ef4b, user_id=e6277cc8ff4841a5a06dbad991b30923, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:21.701274 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeea315-05c8-406d-a145-e41074e6a0ce admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:21.712325 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1272/2511] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:21:21 2022] PUT /identity/v3/projects/26c30a98ebc84c6fb9d4952e10a6ef4b/users/e6277cc8ff4841a5a06dbad991b30923/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:21:21.722748 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f5e2fe-241a-43bf-a929-c9a79215b3e8 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:21.723074 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f5e2fe-241a-43bf-a929-c9a79215b3e8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:21.723357 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f5e2fe-241a-43bf-a929-c9a79215b3e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:21.752079 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-23f5e2fe-241a-43bf-a929-c9a79215b3e8 None None] MFA Rules not processed for user `e6277cc8ff4841a5a06dbad991b30923`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:21:21.753261 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-23f5e2fe-241a-43bf-a929-c9a79215b3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:21.819828 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1240/2512] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:21:21 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:21:21.831248 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeea315-05c8-406d-a145-e41074e6a0ce admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:21.832951 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-feeea315-05c8-406d-a145-e41074e6a0ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:21.891531 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeea315-05c8-406d-a145-e41074e6a0ce admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:21.895147 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045b37b9-8216-4aa3-a499-6e599205030c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:21.895783 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b37b9-8216-4aa3-a499-6e599205030c service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:21.896064 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b37b9-8216-4aa3-a499-6e599205030c service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:21.896339 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045b37b9-8216-4aa3-a499-6e599205030c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:21.905061 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045b37b9-8216-4aa3-a499-6e599205030c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:21.906477 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045b37b9-8216-4aa3-a499-6e599205030c service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:21.916253 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1273/2513] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:21:21 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:21:21.985915 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f5e2fe-241a-43bf-a929-c9a79215b3e8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:22.059163 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f5e2fe-241a-43bf-a929-c9a79215b3e8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:22.066190 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa17538-d044-4402-86e8-f92cc06488cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:22.067686 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa17538-d044-4402-86e8-f92cc06488cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:22.068145 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa17538-d044-4402-86e8-f92cc06488cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:22.068614 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa17538-d044-4402-86e8-f92cc06488cf admin admin] PATH_INFO: `/v3/users/8089293170fa43139ce2c3ad3591ba9a` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:22.070265 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa17538-d044-4402-86e8-f92cc06488cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=8089293170fa43139ce2c3ad3591ba9a)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:22.072270 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa17538-d044-4402-86e8-f92cc06488cf admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:22.084375 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:21:22.129850 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-7fa17538-d044-4402-86e8-f92cc06488cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8089293170fa43139ce2c3ad3591ba9a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:22.136951 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-7fa17538-d044-4402-86e8-f92cc06488cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8089293170fa43139ce2c3ad3591ba9a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:22.146481 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-7fa17538-d044-4402-86e8-f92cc06488cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8089293170fa43139ce2c3ad3591ba9a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:22.147528 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-7fa17538-d044-4402-86e8-f92cc06488cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8089293170fa43139ce2c3ad3591ba9a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:22.148944 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-7fa17538-d044-4402-86e8-f92cc06488cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8089293170fa43139ce2c3ad3591ba9a'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:22.169763 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1241/2514] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:21:21 2022] DELETE /identity/v3/users/8089293170fa43139ce2c3ad3591ba9a => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:21:22.175222 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e1d04-3dff-45c6-b8a6-94379848e98f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:22.175870 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e1d04-3dff-45c6-b8a6-94379848e98f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:22.176352 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e1d04-3dff-45c6-b8a6-94379848e98f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:22.197696 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8f7e1d04-3dff-45c6-b8a6-94379848e98f None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:21:22.199385 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8f7e1d04-3dff-45c6-b8a6-94379848e98f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:22.271666 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1274/2515] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:21:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:21:22.282223 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa17538-d044-4402-86e8-f92cc06488cf admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:22.283682 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7fa17538-d044-4402-86e8-f92cc06488cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:22.346484 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa17538-d044-4402-86e8-f92cc06488cf admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:22.348927 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af0eae-94ec-451d-b580-84af444427d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:22.349581 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af0eae-94ec-451d-b580-84af444427d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:22.349868 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af0eae-94ec-451d-b580-84af444427d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:22.350147 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21af0eae-94ec-451d-b580-84af444427d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:22.363878 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21af0eae-94ec-451d-b580-84af444427d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:22.366659 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21af0eae-94ec-451d-b580-84af444427d9 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:22.381457 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1242/2516] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:21:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:21:22.683073 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65311da5-2e9c-4530-9e10-39fe714d167d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:22.683563 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65311da5-2e9c-4530-9e10-39fe714d167d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:22.684031 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65311da5-2e9c-4530-9e10-39fe714d167d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:22.701115 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af0eae-94ec-451d-b580-84af444427d9 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:22.702779 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-21af0eae-94ec-451d-b580-84af444427d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:22.714366 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-65311da5-2e9c-4530-9e10-39fe714d167d None None] MFA Rules not processed for user `09c23d1c732c452a9597c647ad0241f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:21:22.715640 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-65311da5-2e9c-4530-9e10-39fe714d167d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:22.719725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21af0eae-94ec-451d-b580-84af444427d9 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:22.725947 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6cbecc-54ca-4e43-a640-d34e756e1738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:22.727363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6cbecc-54ca-4e43-a640-d34e756e1738 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:22.727742 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6cbecc-54ca-4e43-a640-d34e756e1738 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:22.727934 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6cbecc-54ca-4e43-a640-d34e756e1738 admin admin] PATH_INFO: `/v3/projects/07142202f39444fa8c3994886a292a91` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:22.729763 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6cbecc-54ca-4e43-a640-d34e756e1738 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07142202f39444fa8c3994886a292a91)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:22.731897 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6cbecc-54ca-4e43-a640-d34e756e1738 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:22.756395 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:21:22.790152 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1275/2517] 10.222.0.50 () {58 vars in 1009 bytes} [Fri May 27 15:21:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:21:22.799401 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ab6cbecc-54ca-4e43-a640-d34e756e1738 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07142202f39444fa8c3994886a292a91', 'request_id': 'req-ab6cbecc-54ca-4e43-a640-d34e756e1738'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:22.806404 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ab6cbecc-54ca-4e43-a640-d34e756e1738 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07142202f39444fa8c3994886a292a91', 'request_id': 'req-ab6cbecc-54ca-4e43-a640-d34e756e1738'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:22.814813 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ab6cbecc-54ca-4e43-a640-d34e756e1738 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07142202f39444fa8c3994886a292a91', 'request_id': 'req-ab6cbecc-54ca-4e43-a640-d34e756e1738'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:22.833831 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ab6cbecc-54ca-4e43-a640-d34e756e1738 admin admin] The token cache is being invalidate because project 07142202f39444fa8c3994886a292a91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:21:22.834147 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ab6cbecc-54ca-4e43-a640-d34e756e1738 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:22.834750 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-ab6cbecc-54ca-4e43-a640-d34e756e1738 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:22.836385 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1243/2518] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:21:22 2022] DELETE /identity/v3/projects/07142202f39444fa8c3994886a292a91 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:21:22.842337 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65311da5-2e9c-4530-9e10-39fe714d167d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:22.849636 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6cbecc-54ca-4e43-a640-d34e756e1738 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:22.852013 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ab6cbecc-54ca-4e43-a640-d34e756e1738 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:22.855648 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65311da5-2e9c-4530-9e10-39fe714d167d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:22.873887 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:21:22.919461 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e13793b-42de-4695-82d4-f647836ecefc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:22.920157 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13793b-42de-4695-82d4-f647836ecefc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:22.920437 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13793b-42de-4695-82d4-f647836ecefc admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:22.920713 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e13793b-42de-4695-82d4-f647836ecefc admin admin] PATH_INFO: `/v3/users/e6277cc8ff4841a5a06dbad991b30923` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:22.922031 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e13793b-42de-4695-82d4-f647836ecefc admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6277cc8ff4841a5a06dbad991b30923)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:22.924070 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e13793b-42de-4695-82d4-f647836ecefc admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:22.941398 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6cbecc-54ca-4e43-a640-d34e756e1738 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:22.947287 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b6ba85-6154-4ea8-9646-c5f9629dfc87 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18903f2eab7c415096e075d47f0b9a4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:22.947934 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6ba85-6154-4ea8-9646-c5f9629dfc87 service swift] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:22.948224 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6ba85-6154-4ea8-9646-c5f9629dfc87 service swift] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:22.948509 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b6ba85-6154-4ea8-9646-c5f9629dfc87 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:22.951381 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-24b6ba85-6154-4ea8-9646-c5f9629dfc87 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:22.986940 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7e13793b-42de-4695-82d4-f647836ecefc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6277cc8ff4841a5a06dbad991b30923'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:22.994959 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7e13793b-42de-4695-82d4-f647836ecefc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6277cc8ff4841a5a06dbad991b30923'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:22.995679 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7e13793b-42de-4695-82d4-f647836ecefc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6277cc8ff4841a5a06dbad991b30923'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:23.003914 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7e13793b-42de-4695-82d4-f647836ecefc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6277cc8ff4841a5a06dbad991b30923'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:23.004540 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7e13793b-42de-4695-82d4-f647836ecefc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6277cc8ff4841a5a06dbad991b30923'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:23.011951 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b6ba85-6154-4ea8-9646-c5f9629dfc87 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:23.013599 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b6ba85-6154-4ea8-9646-c5f9629dfc87 service swift] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:23.021655 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1244/2519] 10.222.0.50 () {60 vars in 1384 bytes} [Fri May 27 15:21:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 27 15:21:23.023475 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1276/2520] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:21:22 2022] DELETE /identity/v3/users/e6277cc8ff4841a5a06dbad991b30923 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:21:23.449897 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b6ba85-6154-4ea8-9646-c5f9629dfc87 service swift] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:23.451457 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-24b6ba85-6154-4ea8-9646-c5f9629dfc87 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:23.468388 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b6ba85-6154-4ea8-9646-c5f9629dfc87 service swift] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:23.552149 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bc9799-6720-4615-9207-b6adfaeeb4f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:23.553232 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc9799-6720-4615-9207-b6adfaeeb4f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:23.553529 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc9799-6720-4615-9207-b6adfaeeb4f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:23.554328 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bc9799-6720-4615-9207-b6adfaeeb4f7 admin admin] PATH_INFO: `/v3/projects/26c30a98ebc84c6fb9d4952e10a6ef4b` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:23.555718 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bc9799-6720-4615-9207-b6adfaeeb4f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26c30a98ebc84c6fb9d4952e10a6ef4b)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:23.558489 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bc9799-6720-4615-9207-b6adfaeeb4f7 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:23.639369 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-63bc9799-6720-4615-9207-b6adfaeeb4f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26c30a98ebc84c6fb9d4952e10a6ef4b', 'request_id': 'req-63bc9799-6720-4615-9207-b6adfaeeb4f7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:23.648348 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:21:23.657178 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-63bc9799-6720-4615-9207-b6adfaeeb4f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26c30a98ebc84c6fb9d4952e10a6ef4b', 'request_id': 'req-63bc9799-6720-4615-9207-b6adfaeeb4f7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:23.668232 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-63bc9799-6720-4615-9207-b6adfaeeb4f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26c30a98ebc84c6fb9d4952e10a6ef4b', 'request_id': 'req-63bc9799-6720-4615-9207-b6adfaeeb4f7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:23.692662 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-63bc9799-6720-4615-9207-b6adfaeeb4f7 admin admin] The token cache is being invalidate because project 26c30a98ebc84c6fb9d4952e10a6ef4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:21:23.692919 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-63bc9799-6720-4615-9207-b6adfaeeb4f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:23.693610 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-63bc9799-6720-4615-9207-b6adfaeeb4f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:23.696513 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1245/2521] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:21:23 2022] DELETE /identity/v3/projects/26c30a98ebc84c6fb9d4952e10a6ef4b => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:21:23.701864 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929c739-54c5-49d2-b9bd-d2c8e1567ed7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:23.702466 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929c739-54c5-49d2-b9bd-d2c8e1567ed7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:23.702818 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9929c739-54c5-49d2-b9bd-d2c8e1567ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:23.726901 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9929c739-54c5-49d2-b9bd-d2c8e1567ed7 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:21:23.730383 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9929c739-54c5-49d2-b9bd-d2c8e1567ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:23.815346 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1277/2522] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:21:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:21:23.823992 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d5b5d-16ef-4329-b12e-1da0c303d4e0 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:23.824928 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d5b5d-16ef-4329-b12e-1da0c303d4e0 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:23.825213 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d5b5d-16ef-4329-b12e-1da0c303d4e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:23.847627 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-183d5b5d-16ef-4329-b12e-1da0c303d4e0 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:21:23.849306 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-183d5b5d-16ef-4329-b12e-1da0c303d4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:23.859463 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1246/2523] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:21:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:21:23.867378 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9929c739-54c5-49d2-b9bd-d2c8e1567ed7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:23.873890 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:21:23.878773 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9929c739-54c5-49d2-b9bd-d2c8e1567ed7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:23.893573 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7496548-dc76-471c-9fa4-41a9b31eed2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:23.894524 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7496548-dc76-471c-9fa4-41a9b31eed2a admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:23.895002 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7496548-dc76-471c-9fa4-41a9b31eed2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:23.895583 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7496548-dc76-471c-9fa4-41a9b31eed2a admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:23.896657 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7496548-dc76-471c-9fa4-41a9b31eed2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:21:23.898772 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7496548-dc76-471c-9fa4-41a9b31eed2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:23.903559 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7496548-dc76-471c-9fa4-41a9b31eed2a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:23.930444 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1278/2524] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:21:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:21:23.930938 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183d5b5d-16ef-4329-b12e-1da0c303d4e0 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:23.942329 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183d5b5d-16ef-4329-b12e-1da0c303d4e0 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:23.959075 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057f4168-e12c-4e1b-a8f8-6aae8cc3c094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:23.959934 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f4168-e12c-4e1b-a8f8-6aae8cc3c094 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:23.961146 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f4168-e12c-4e1b-a8f8-6aae8cc3c094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:23.961633 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f4168-e12c-4e1b-a8f8-6aae8cc3c094 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:23.962551 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057f4168-e12c-4e1b-a8f8-6aae8cc3c094 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:23.964826 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057f4168-e12c-4e1b-a8f8-6aae8cc3c094 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:24.008677 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1247/2525] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:21:23 2022] POST /identity/v3/projects => generated 373 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:21:24.013341 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7496548-dc76-471c-9fa4-41a9b31eed2a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:24.026350 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7496548-dc76-471c-9fa4-41a9b31eed2a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:24.110864 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aee9f7-1d49-478e-8384-ccafdcd2166d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:24.111879 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aee9f7-1d49-478e-8384-ccafdcd2166d admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:24.112233 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aee9f7-1d49-478e-8384-ccafdcd2166d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:24.112825 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aee9f7-1d49-478e-8384-ccafdcd2166d admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:24.113835 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aee9f7-1d49-478e-8384-ccafdcd2166d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:24.115903 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aee9f7-1d49-478e-8384-ccafdcd2166d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:24.149881 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057f4168-e12c-4e1b-a8f8-6aae8cc3c094 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:24.150332 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1279/2526] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:21:24 2022] POST /identity/v3/users => generated 287 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:21:24.165890 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057f4168-e12c-4e1b-a8f8-6aae8cc3c094 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:24.180146 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942064b9-8072-4b6e-8434-c0078c993ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:24.180851 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942064b9-8072-4b6e-8434-c0078c993ad1 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:24.181143 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942064b9-8072-4b6e-8434-c0078c993ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:24.181432 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942064b9-8072-4b6e-8434-c0078c993ad1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:24.182618 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942064b9-8072-4b6e-8434-c0078c993ad1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:24.185624 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942064b9-8072-4b6e-8434-c0078c993ad1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:24.214997 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1248/2527] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:21:24 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:21:24.218528 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aee9f7-1d49-478e-8384-ccafdcd2166d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:24.232502 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aee9f7-1d49-478e-8384-ccafdcd2166d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:24.242128 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f011ba3-3d78-4df4-bc12-f8de9e361125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:24.242897 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f011ba3-3d78-4df4-bc12-f8de9e361125 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:24.243190 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f011ba3-3d78-4df4-bc12-f8de9e361125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:24.243470 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f011ba3-3d78-4df4-bc12-f8de9e361125 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:24.244199 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f011ba3-3d78-4df4-bc12-f8de9e361125 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:24.245857 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f011ba3-3d78-4df4-bc12-f8de9e361125 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:24.257610 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1280/2528] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:21:24 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:21:24.261710 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942064b9-8072-4b6e-8434-c0078c993ad1 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:24.273943 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942064b9-8072-4b6e-8434-c0078c993ad1 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:24.278842 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ae41db-9edf-4b3b-b83e-2379c3607b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:24.279815 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae41db-9edf-4b3b-b83e-2379c3607b1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:24.280194 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae41db-9edf-4b3b-b83e-2379c3607b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:24.280476 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ae41db-9edf-4b3b-b83e-2379c3607b1b admin admin] PATH_INFO: `/v3/projects/3f64c5cccee042209e76ad531fa84900/users/71634942c5df4885987839adc5c4d092/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:24.299782 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ae41db-9edf-4b3b-b83e-2379c3607b1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f64c5cccee042209e76ad531fa84900, user_id=71634942c5df4885987839adc5c4d092, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:24.302672 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ae41db-9edf-4b3b-b83e-2379c3607b1b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:24.326918 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1249/2529] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:21:24 2022] PUT /identity/v3/projects/3f64c5cccee042209e76ad531fa84900/users/71634942c5df4885987839adc5c4d092/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:21:24.332002 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df0861f-32de-4f24-b5c5-1337759c2d36 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:24.332326 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df0861f-32de-4f24-b5c5-1337759c2d36 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:24.332608 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df0861f-32de-4f24-b5c5-1337759c2d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:24.358648 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8df0861f-32de-4f24-b5c5-1337759c2d36 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:21:24.360994 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8df0861f-32de-4f24-b5c5-1337759c2d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:24.444920 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1281/2530] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:21:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:21:24.454324 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ae41db-9edf-4b3b-b83e-2379c3607b1b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:24.456338 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-95ae41db-9edf-4b3b-b83e-2379c3607b1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:24.561127 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ae41db-9edf-4b3b-b83e-2379c3607b1b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:24.571582 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f01c5-24b1-4c22-bfb7-91b73d924737 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:24.572259 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f01c5-24b1-4c22-bfb7-91b73d924737 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:24.572548 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f01c5-24b1-4c22-bfb7-91b73d924737 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:24.572827 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f01c5-24b1-4c22-bfb7-91b73d924737 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:24.592338 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31f01c5-24b1-4c22-bfb7-91b73d924737 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:24.595406 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31f01c5-24b1-4c22-bfb7-91b73d924737 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:24.606921 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1250/2531] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:21:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:21:25.464268 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f70756f-3874-4635-9eb5-4ef61a557be2 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:25.464734 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f70756f-3874-4635-9eb5-4ef61a557be2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:25.465054 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f70756f-3874-4635-9eb5-4ef61a557be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:25.497783 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4f70756f-3874-4635-9eb5-4ef61a557be2 None None] MFA Rules not processed for user `71634942c5df4885987839adc5c4d092`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:21:25.499221 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4f70756f-3874-4635-9eb5-4ef61a557be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:25.585805 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:21:25.591310 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1282/2532] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:21:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:21:25.595802 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f01c5-24b1-4c22-bfb7-91b73d924737 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:25.613702 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f01c5-24b1-4c22-bfb7-91b73d924737 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:25.622356 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0e4105-065d-49db-9577-65704c10dd28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:25.623053 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e4105-065d-49db-9577-65704c10dd28 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:25.623337 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e4105-065d-49db-9577-65704c10dd28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:25.623615 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0e4105-065d-49db-9577-65704c10dd28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:25.624314 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0e4105-065d-49db-9577-65704c10dd28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:25.628028 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0e4105-065d-49db-9577-65704c10dd28 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:25.700629 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1251/2533] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:21:25 2022] POST /identity/v3/projects => generated 371 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:21:25.704886 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f70756f-3874-4635-9eb5-4ef61a557be2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:25.721528 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f70756f-3874-4635-9eb5-4ef61a557be2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:25.799995 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd730e7-747e-4b4e-9926-28ab2738d528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:25.801125 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd730e7-747e-4b4e-9926-28ab2738d528 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:25.801516 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd730e7-747e-4b4e-9926-28ab2738d528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:25.801900 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd730e7-747e-4b4e-9926-28ab2738d528 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:25.802811 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd730e7-747e-4b4e-9926-28ab2738d528 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:25.807437 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd730e7-747e-4b4e-9926-28ab2738d528 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:25.853037 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1283/2534] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:21:25 2022] POST /identity/v3/users => generated 292 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:21:25.856765 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0e4105-065d-49db-9577-65704c10dd28 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:25.872637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0e4105-065d-49db-9577-65704c10dd28 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:25.878829 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19939d81-5150-4dce-996f-074538ca2f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:25.880243 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19939d81-5150-4dce-996f-074538ca2f4c admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:25.880750 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19939d81-5150-4dce-996f-074538ca2f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:25.881301 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19939d81-5150-4dce-996f-074538ca2f4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:25.882825 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19939d81-5150-4dce-996f-074538ca2f4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:25.884976 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19939d81-5150-4dce-996f-074538ca2f4c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:25.897471 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1252/2535] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:21:25 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:21:25.901553 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd730e7-747e-4b4e-9926-28ab2738d528 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:25.916563 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd730e7-747e-4b4e-9926-28ab2738d528 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:25.932168 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9594b2-f599-4c7b-a311-6679b7adc464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:25.933246 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9594b2-f599-4c7b-a311-6679b7adc464 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:25.933637 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9594b2-f599-4c7b-a311-6679b7adc464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:25.934018 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9594b2-f599-4c7b-a311-6679b7adc464 admin admin] PATH_INFO: `/v3/projects/4b34ff761e1f412386285ff2d8142a6b/users/25c8c23dbf3143ab88176662021b4e30/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:25.968902 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9594b2-f599-4c7b-a311-6679b7adc464 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b34ff761e1f412386285ff2d8142a6b, user_id=25c8c23dbf3143ab88176662021b4e30, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:25.974021 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9594b2-f599-4c7b-a311-6679b7adc464 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:26.008870 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1284/2536] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:21:25 2022] PUT /identity/v3/projects/4b34ff761e1f412386285ff2d8142a6b/users/25c8c23dbf3143ab88176662021b4e30/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:21:26.012876 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19939d81-5150-4dce-996f-074538ca2f4c admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:26.027667 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19939d81-5150-4dce-996f-074538ca2f4c admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:26.088697 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:21:26.103328 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09b9e84-593d-4dc7-b933-d51abd137aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:26.104027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b9e84-593d-4dc7-b933-d51abd137aff admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:26.104311 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b9e84-593d-4dc7-b933-d51abd137aff admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:26.104588 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09b9e84-593d-4dc7-b933-d51abd137aff admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:26.105322 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09b9e84-593d-4dc7-b933-d51abd137aff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:26.112838 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09b9e84-593d-4dc7-b933-d51abd137aff admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:26.133268 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1253/2537] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:21:26 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:21:26.137451 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9594b2-f599-4c7b-a311-6679b7adc464 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:26.149466 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9594b2-f599-4c7b-a311-6679b7adc464 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:26.158439 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e911d69d-1295-4051-831b-81ac06da21ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:26.158439 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e911d69d-1295-4051-831b-81ac06da21ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:26.158439 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e911d69d-1295-4051-831b-81ac06da21ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:26.158439 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e911d69d-1295-4051-831b-81ac06da21ac admin admin] PATH_INFO: `/v3/projects/4b34ff761e1f412386285ff2d8142a6b/users/25c8c23dbf3143ab88176662021b4e30/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:26.159228 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e911d69d-1295-4051-831b-81ac06da21ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b34ff761e1f412386285ff2d8142a6b, user_id=25c8c23dbf3143ab88176662021b4e30, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:26.161928 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e911d69d-1295-4051-831b-81ac06da21ac admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:26.174227 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1285/2538] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:21:26 2022] PUT /identity/v3/projects/4b34ff761e1f412386285ff2d8142a6b/users/25c8c23dbf3143ab88176662021b4e30/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:21:26.999023 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd2faac-76b9-4d08-b1ac-153a4a6f5129 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:26.999860 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd2faac-76b9-4d08-b1ac-153a4a6f5129 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:27.000471 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd2faac-76b9-4d08-b1ac-153a4a6f5129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:27.045761 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7cd2faac-76b9-4d08-b1ac-153a4a6f5129 None None] MFA Rules not processed for user `25c8c23dbf3143ab88176662021b4e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:21:27.047212 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7cd2faac-76b9-4d08-b1ac-153a4a6f5129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:27.137904 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1254/2539] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:21:26 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:21:27.149893 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e911d69d-1295-4051-831b-81ac06da21ac admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:27.152346 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e911d69d-1295-4051-831b-81ac06da21ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:27.237497 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e911d69d-1295-4051-831b-81ac06da21ac admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:27.242505 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e3fe1-7a97-4cb2-9c6b-4240aa0f297f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:27.243341 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e3fe1-7a97-4cb2-9c6b-4240aa0f297f service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:27.243734 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e3fe1-7a97-4cb2-9c6b-4240aa0f297f service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:27.244715 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e3fe1-7a97-4cb2-9c6b-4240aa0f297f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:27.345546 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4e3fe1-7a97-4cb2-9c6b-4240aa0f297f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:27.348677 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4e3fe1-7a97-4cb2-9c6b-4240aa0f297f service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:27.361752 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1286/2540] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:21:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:21:28.776677 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd2faac-76b9-4d08-b1ac-153a4a6f5129 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:28.786465 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd2faac-76b9-4d08-b1ac-153a4a6f5129 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:28.792618 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27db2cd0-9742-464e-a3f8-45b1094748f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:28.793359 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db2cd0-9742-464e-a3f8-45b1094748f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:28.793710 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db2cd0-9742-464e-a3f8-45b1094748f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:28.794078 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db2cd0-9742-464e-a3f8-45b1094748f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:28.804723 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27db2cd0-9742-464e-a3f8-45b1094748f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:28.806431 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27db2cd0-9742-464e-a3f8-45b1094748f7 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:28.818874 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1255/2541] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:21:28 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:21:37.356086 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:21:38.812183 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=46 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:21:41.287083 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca39d130-b26a-420a-b863-4557a713b118 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:41.288225 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca39d130-b26a-420a-b863-4557a713b118 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:41.288225 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca39d130-b26a-420a-b863-4557a713b118 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:41.289552 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1287/2542] 10.222.0.50 () {54 vars in 921 bytes} [Fri May 27 15:21:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:21:41.297277 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c153e924-849f-4adc-baa4-b1ed44c51c50 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:41.297798 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c153e924-849f-4adc-baa4-b1ed44c51c50 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:41.297898 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c153e924-849f-4adc-baa4-b1ed44c51c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:41.318843 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c153e924-849f-4adc-baa4-b1ed44c51c50 None None] MFA Rules not processed for user `451a9232f95e494683d81b6078a318d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:21:41.320596 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c153e924-849f-4adc-baa4-b1ed44c51c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:41.437298 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1256/2543] 10.222.0.50 () {58 vars in 1018 bytes} [Fri May 27 15:21:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:21:41.444796 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca39d130-b26a-420a-b863-4557a713b118 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:41.448179 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ca39d130-b26a-420a-b863-4557a713b118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:41.525918 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca39d130-b26a-420a-b863-4557a713b118 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:41.529962 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33923048-3315-4015-9ffc-a01917f523cf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1afda9d6b1d64e6ab400b26124cf1290', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:41.530706 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33923048-3315-4015-9ffc-a01917f523cf service glance] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:41.531076 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33923048-3315-4015-9ffc-a01917f523cf service glance] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:41.531537 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33923048-3315-4015-9ffc-a01917f523cf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:41.543427 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33923048-3315-4015-9ffc-a01917f523cf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:41.545300 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33923048-3315-4015-9ffc-a01917f523cf service glance] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:41.558690 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1288/2544] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:21:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:21:41.638899 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab8231-ae00-4450-8726-f5ed5f5afe71 None None] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:41.639418 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab8231-ae00-4450-8726-f5ed5f5afe71 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:41.639701 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab8231-ae00-4450-8726-f5ed5f5afe71 None None] PATH_INFO: `/` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:41.641091 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1257/2545] 10.222.0.50 () {54 vars in 921 bytes} [Fri May 27 15:21:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:21:41.651069 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64913e1-565d-4226-91c3-c8e255ecf88b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:41.651575 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64913e1-565d-4226-91c3-c8e255ecf88b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:41.651954 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64913e1-565d-4226-91c3-c8e255ecf88b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:41.679920 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b64913e1-565d-4226-91c3-c8e255ecf88b None None] MFA Rules not processed for user `f874edc08438434189f1d14d79291487`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:21:41.681358 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b64913e1-565d-4226-91c3-c8e255ecf88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:41.747320 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1289/2546] 10.222.0.50 () {58 vars in 1018 bytes} [Fri May 27 15:21:41 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:21:41.761008 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab8231-ae00-4450-8726-f5ed5f5afe71 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:41.777349 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab8231-ae00-4450-8726-f5ed5f5afe71 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:41.781177 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f9120b-eaef-4612-9990-059c0b2dbdc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:41.781818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9120b-eaef-4612-9990-059c0b2dbdc8 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:41.782139 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9120b-eaef-4612-9990-059c0b2dbdc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:41.782422 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9120b-eaef-4612-9990-059c0b2dbdc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:41.791515 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f9120b-eaef-4612-9990-059c0b2dbdc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:41.793048 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f9120b-eaef-4612-9990-059c0b2dbdc8 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:41.811138 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1258/2547] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:21:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:21:41.901589 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66f470-abee-4a81-970f-1e53067b8bbf None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:41.901881 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66f470-abee-4a81-970f-1e53067b8bbf None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:41.902162 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66f470-abee-4a81-970f-1e53067b8bbf None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:41.903651 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1290/2548] 10.222.0.50 () {54 vars in 921 bytes} [Fri May 27 15:21:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 27 15:21:41.909697 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb966459-c24f-49c4-aa8c-4d6524812488 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:41.910061 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb966459-c24f-49c4-aa8c-4d6524812488 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:41.910404 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb966459-c24f-49c4-aa8c-4d6524812488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:41.929266 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-cb966459-c24f-49c4-aa8c-4d6524812488 None None] MFA Rules not processed for user `451a9232f95e494683d81b6078a318d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:21:41.930511 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cb966459-c24f-49c4-aa8c-4d6524812488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:41.940794 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1259/2549] 10.222.0.50 () {58 vars in 1018 bytes} [Fri May 27 15:21:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:21:41.951853 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c66f470-abee-4a81-970f-1e53067b8bbf None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:41.963134 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c66f470-abee-4a81-970f-1e53067b8bbf None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:41.966726 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d492823a-1c00-49a5-a21d-708a18ed27d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:41.967468 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492823a-1c00-49a5-a21d-708a18ed27d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:41.967854 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492823a-1c00-49a5-a21d-708a18ed27d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:41.968201 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d492823a-1c00-49a5-a21d-708a18ed27d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:41.979722 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d492823a-1c00-49a5-a21d-708a18ed27d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:41.981648 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d492823a-1c00-49a5-a21d-708a18ed27d6 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:41.994179 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1291/2550] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:21:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:21:42.858691 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb966459-c24f-49c4-aa8c-4d6524812488 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:42.929764 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb966459-c24f-49c4-aa8c-4d6524812488 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:42.935291 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67486205-3c68-4f1d-b26b-3e89f6ebdf8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:42.936206 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67486205-3c68-4f1d-b26b-3e89f6ebdf8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:42.936532 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67486205-3c68-4f1d-b26b-3e89f6ebdf8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:42.936802 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67486205-3c68-4f1d-b26b-3e89f6ebdf8e admin admin] PATH_INFO: `/v3/users/71634942c5df4885987839adc5c4d092` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:42.938004 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67486205-3c68-4f1d-b26b-3e89f6ebdf8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=71634942c5df4885987839adc5c4d092)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:42.940072 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67486205-3c68-4f1d-b26b-3e89f6ebdf8e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:42.995797 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-67486205-3c68-4f1d-b26b-3e89f6ebdf8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71634942c5df4885987839adc5c4d092'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:43.003870 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-67486205-3c68-4f1d-b26b-3e89f6ebdf8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71634942c5df4885987839adc5c4d092'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:43.011514 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-67486205-3c68-4f1d-b26b-3e89f6ebdf8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71634942c5df4885987839adc5c4d092'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:43.012219 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-67486205-3c68-4f1d-b26b-3e89f6ebdf8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71634942c5df4885987839adc5c4d092'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:43.012865 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-67486205-3c68-4f1d-b26b-3e89f6ebdf8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71634942c5df4885987839adc5c4d092'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:43.020993 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:21:43.033101 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1260/2551] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:21:42 2022] DELETE /identity/v3/users/71634942c5df4885987839adc5c4d092 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:21:43.215345 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d492823a-1c00-49a5-a21d-708a18ed27d6 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:43.228280 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d492823a-1c00-49a5-a21d-708a18ed27d6 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:43.315785 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5673fdb3-750c-4a9a-8469-3263a19e6dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:43.316907 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5673fdb3-750c-4a9a-8469-3263a19e6dc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:43.317187 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5673fdb3-750c-4a9a-8469-3263a19e6dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:43.317574 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5673fdb3-750c-4a9a-8469-3263a19e6dc2 admin admin] PATH_INFO: `/v3/projects/3f64c5cccee042209e76ad531fa84900` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:43.320173 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5673fdb3-750c-4a9a-8469-3263a19e6dc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f64c5cccee042209e76ad531fa84900)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:43.322576 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5673fdb3-750c-4a9a-8469-3263a19e6dc2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:43.371006 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:21:43.384985 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5673fdb3-750c-4a9a-8469-3263a19e6dc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f64c5cccee042209e76ad531fa84900', 'request_id': 'req-5673fdb3-750c-4a9a-8469-3263a19e6dc2'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:43.394498 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5673fdb3-750c-4a9a-8469-3263a19e6dc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f64c5cccee042209e76ad531fa84900', 'request_id': 'req-5673fdb3-750c-4a9a-8469-3263a19e6dc2'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:43.401652 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5673fdb3-750c-4a9a-8469-3263a19e6dc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f64c5cccee042209e76ad531fa84900', 'request_id': 'req-5673fdb3-750c-4a9a-8469-3263a19e6dc2'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:43.417231 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5673fdb3-750c-4a9a-8469-3263a19e6dc2 admin admin] The token cache is being invalidate because project 3f64c5cccee042209e76ad531fa84900 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:21:43.417564 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5673fdb3-750c-4a9a-8469-3263a19e6dc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:43.419037 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5673fdb3-750c-4a9a-8469-3263a19e6dc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:43.420517 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1292/2552] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:21:43 2022] DELETE /identity/v3/projects/3f64c5cccee042209e76ad531fa84900 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:21:43.424299 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67486205-3c68-4f1d-b26b-3e89f6ebdf8e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:43.425918 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-67486205-3c68-4f1d-b26b-3e89f6ebdf8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:43.440626 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67486205-3c68-4f1d-b26b-3e89f6ebdf8e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:43.516064 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8762b750-70b0-4f86-8bc6-fdb3b0bafb75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:43.517008 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8762b750-70b0-4f86-8bc6-fdb3b0bafb75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:43.517300 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8762b750-70b0-4f86-8bc6-fdb3b0bafb75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:43.518063 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8762b750-70b0-4f86-8bc6-fdb3b0bafb75 admin admin] PATH_INFO: `/v3/users/25c8c23dbf3143ab88176662021b4e30` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:43.518957 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8762b750-70b0-4f86-8bc6-fdb3b0bafb75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25c8c23dbf3143ab88176662021b4e30)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:43.520939 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8762b750-70b0-4f86-8bc6-fdb3b0bafb75 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:43.576785 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8762b750-70b0-4f86-8bc6-fdb3b0bafb75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25c8c23dbf3143ab88176662021b4e30'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:43.584670 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8762b750-70b0-4f86-8bc6-fdb3b0bafb75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25c8c23dbf3143ab88176662021b4e30'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:43.592342 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8762b750-70b0-4f86-8bc6-fdb3b0bafb75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25c8c23dbf3143ab88176662021b4e30'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:43.593065 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8762b750-70b0-4f86-8bc6-fdb3b0bafb75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25c8c23dbf3143ab88176662021b4e30'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:43.593761 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-8762b750-70b0-4f86-8bc6-fdb3b0bafb75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25c8c23dbf3143ab88176662021b4e30'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:43.613120 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1261/2553] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:21:43 2022] DELETE /identity/v3/users/25c8c23dbf3143ab88176662021b4e30 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:21:43.794070 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5673fdb3-750c-4a9a-8469-3263a19e6dc2 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:43.795692 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5673fdb3-750c-4a9a-8469-3263a19e6dc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:43.815115 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5673fdb3-750c-4a9a-8469-3263a19e6dc2 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:43.881216 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17912c31-883e-45e4-9dad-b98ad1a17c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:43.882089 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17912c31-883e-45e4-9dad-b98ad1a17c90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:43.882374 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17912c31-883e-45e4-9dad-b98ad1a17c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:43.882651 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17912c31-883e-45e4-9dad-b98ad1a17c90 admin admin] PATH_INFO: `/v3/projects/4b34ff761e1f412386285ff2d8142a6b` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:43.883865 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17912c31-883e-45e4-9dad-b98ad1a17c90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b34ff761e1f412386285ff2d8142a6b)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:43.885769 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17912c31-883e-45e4-9dad-b98ad1a17c90 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:43.905177 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9beac-8042-4e39-b5cd-5b3813b3d740 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:43.905690 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9beac-8042-4e39-b5cd-5b3813b3d740 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:43.905791 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9beac-8042-4e39-b5cd-5b3813b3d740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:43.926448 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c1f9beac-8042-4e39-b5cd-5b3813b3d740 None None] MFA Rules not processed for user `09c23d1c732c452a9597c647ad0241f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:21:43.927587 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c1f9beac-8042-4e39-b5cd-5b3813b3d740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:43.939866 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-17912c31-883e-45e4-9dad-b98ad1a17c90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b34ff761e1f412386285ff2d8142a6b', 'request_id': 'req-17912c31-883e-45e4-9dad-b98ad1a17c90'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:43.950023 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-17912c31-883e-45e4-9dad-b98ad1a17c90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b34ff761e1f412386285ff2d8142a6b', 'request_id': 'req-17912c31-883e-45e4-9dad-b98ad1a17c90'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:43.957632 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-17912c31-883e-45e4-9dad-b98ad1a17c90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b34ff761e1f412386285ff2d8142a6b', 'request_id': 'req-17912c31-883e-45e4-9dad-b98ad1a17c90'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:43.973234 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-17912c31-883e-45e4-9dad-b98ad1a17c90 admin admin] The token cache is being invalidate because project 4b34ff761e1f412386285ff2d8142a6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:21:43.973765 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-17912c31-883e-45e4-9dad-b98ad1a17c90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:43.976797 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-17912c31-883e-45e4-9dad-b98ad1a17c90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:43.983836 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1293/2554] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:21:43 2022] DELETE /identity/v3/projects/4b34ff761e1f412386285ff2d8142a6b => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:21:43.985594 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c0ae9-7c62-4973-8a39-08efd5494ae6 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:43.987026 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c0ae9-7c62-4973-8a39-08efd5494ae6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:43.988859 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c0ae9-7c62-4973-8a39-08efd5494ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:44.023036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1262/2555] 10.222.0.50 () {58 vars in 1009 bytes} [Fri May 27 15:21:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:21:44.034002 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f9beac-8042-4e39-b5cd-5b3813b3d740 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:44.034834 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-227c0ae9-7c62-4973-8a39-08efd5494ae6 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:21:44.035711 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c1f9beac-8042-4e39-b5cd-5b3813b3d740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:44.036495 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-227c0ae9-7c62-4973-8a39-08efd5494ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:44.056823 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:21:44.116596 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:21:44.127544 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1294/2556] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:21:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:21:44.129214 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f9beac-8042-4e39-b5cd-5b3813b3d740 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:44.134776 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e8afd2-50fe-46e1-929f-555adfbcb22f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18903f2eab7c415096e075d47f0b9a4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:44.135466 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8afd2-50fe-46e1-929f-555adfbcb22f service swift] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:44.135748 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8afd2-50fe-46e1-929f-555adfbcb22f service swift] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:44.136030 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e8afd2-50fe-46e1-929f-555adfbcb22f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:44.137181 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d2137c-24e8-4549-98a1-51f100378f82 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:44.137601 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d2137c-24e8-4549-98a1-51f100378f82 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:44.138089 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d2137c-24e8-4549-98a1-51f100378f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:44.145212 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e8afd2-50fe-46e1-929f-555adfbcb22f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:44.147709 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e8afd2-50fe-46e1-929f-555adfbcb22f service swift] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:44.158386 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b2d2137c-24e8-4549-98a1-51f100378f82 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:21:44.159109 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1263/2557] 10.222.0.50 () {60 vars in 1384 bytes} [Fri May 27 15:21:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 15:21:44.164341 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b2d2137c-24e8-4549-98a1-51f100378f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:44.175820 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1295/2558] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:21:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:21:44.180071 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e8afd2-50fe-46e1-929f-555adfbcb22f service swift] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:44.195359 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e8afd2-50fe-46e1-929f-555adfbcb22f service swift] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:44.200321 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c63f90-a22d-45f2-884a-b5084b5f3490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:44.200993 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c63f90-a22d-45f2-884a-b5084b5f3490 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:44.201285 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c63f90-a22d-45f2-884a-b5084b5f3490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:44.201561 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c63f90-a22d-45f2-884a-b5084b5f3490 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:44.203252 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c63f90-a22d-45f2-884a-b5084b5f3490 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:21:44.203252 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c63f90-a22d-45f2-884a-b5084b5f3490 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:44.204235 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c63f90-a22d-45f2-884a-b5084b5f3490 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:44.223297 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1264/2559] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:21:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:21:44.224078 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d2137c-24e8-4549-98a1-51f100378f82 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:44.238640 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d2137c-24e8-4549-98a1-51f100378f82 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:44.257667 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e063b671-becd-499f-ab9d-004349a005b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:44.258832 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e063b671-becd-499f-ab9d-004349a005b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:44.259466 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e063b671-becd-499f-ab9d-004349a005b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:44.259961 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e063b671-becd-499f-ab9d-004349a005b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:44.260891 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e063b671-becd-499f-ab9d-004349a005b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:44.264988 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e063b671-becd-499f-ab9d-004349a005b4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:44.322592 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c63f90-a22d-45f2-884a-b5084b5f3490 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:44.327954 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1296/2560] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:21:44 2022] POST /identity/v3/projects => generated 383 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:21:44.335853 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c63f90-a22d-45f2-884a-b5084b5f3490 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:44.403050 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b9f681-c68b-4105-aac9-c4a419dc651d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:44.404118 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9f681-c68b-4105-aac9-c4a419dc651d admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:44.404619 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9f681-c68b-4105-aac9-c4a419dc651d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:44.405134 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b9f681-c68b-4105-aac9-c4a419dc651d admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:44.406099 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b9f681-c68b-4105-aac9-c4a419dc651d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:44.408543 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b9f681-c68b-4105-aac9-c4a419dc651d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:44.435489 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1265/2561] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:21:44 2022] POST /identity/v3/users => generated 292 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:21:44.440549 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e063b671-becd-499f-ab9d-004349a005b4 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:44.456564 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e063b671-becd-499f-ab9d-004349a005b4 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:44.462709 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a10e449-67fd-488f-8c63-9084bc6adbd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:44.463898 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10e449-67fd-488f-8c63-9084bc6adbd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:44.464279 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10e449-67fd-488f-8c63-9084bc6adbd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:44.464705 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a10e449-67fd-488f-8c63-9084bc6adbd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:44.465619 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a10e449-67fd-488f-8c63-9084bc6adbd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:44.467824 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a10e449-67fd-488f-8c63-9084bc6adbd0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:44.480882 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1297/2562] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:21:44.484680 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b9f681-c68b-4105-aac9-c4a419dc651d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:44.496628 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b9f681-c68b-4105-aac9-c4a419dc651d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:44.503228 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a23758-4e0b-4565-9409-5975f4b50e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:44.503903 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a23758-4e0b-4565-9409-5975f4b50e90 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:44.504293 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a23758-4e0b-4565-9409-5975f4b50e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:44.504677 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a23758-4e0b-4565-9409-5975f4b50e90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:44.505709 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a23758-4e0b-4565-9409-5975f4b50e90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:44.508192 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a23758-4e0b-4565-9409-5975f4b50e90 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:44.532771 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1266/2563] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:21:44.534272 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a10e449-67fd-488f-8c63-9084bc6adbd0 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:44.549439 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a10e449-67fd-488f-8c63-9084bc6adbd0 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:44.556836 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43bd300-257c-4530-b063-9a842168959c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:44.558320 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43bd300-257c-4530-b063-9a842168959c admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:44.558935 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43bd300-257c-4530-b063-9a842168959c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:44.559486 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43bd300-257c-4530-b063-9a842168959c admin admin] PATH_INFO: `/v3/projects/eb3f920d163c4bfa804cb6a1a8f17696/users/4d2dd262ed2649e5b33c0e95cb5811cf/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:44.582292 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43bd300-257c-4530-b063-9a842168959c admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb3f920d163c4bfa804cb6a1a8f17696, user_id=4d2dd262ed2649e5b33c0e95cb5811cf, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:44.586225 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43bd300-257c-4530-b063-9a842168959c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:44.622294 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1298/2564] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:21:44 2022] PUT /identity/v3/projects/eb3f920d163c4bfa804cb6a1a8f17696/users/4d2dd262ed2649e5b33c0e95cb5811cf/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:21:44.635099 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659c1a2-551d-4123-8a48-074181cdab2b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:44.635534 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659c1a2-551d-4123-8a48-074181cdab2b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:44.635916 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7659c1a2-551d-4123-8a48-074181cdab2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:44.683124 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7659c1a2-551d-4123-8a48-074181cdab2b None None] MFA Rules not processed for user `4d2dd262ed2649e5b33c0e95cb5811cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:21:44.684298 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7659c1a2-551d-4123-8a48-074181cdab2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:44.776579 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:21:44.781111 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1267/2565] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:21:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:21:44.795950 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43bd300-257c-4530-b063-9a842168959c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:44.797238 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a43bd300-257c-4530-b063-9a842168959c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:44.862235 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43bd300-257c-4530-b063-9a842168959c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:44.866400 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b664e1e-b488-46d8-90ab-7894cae74113 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:44.867151 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b664e1e-b488-46d8-90ab-7894cae74113 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:44.867435 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b664e1e-b488-46d8-90ab-7894cae74113 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:44.867715 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b664e1e-b488-46d8-90ab-7894cae74113 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:44.876756 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b664e1e-b488-46d8-90ab-7894cae74113 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:44.878187 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b664e1e-b488-46d8-90ab-7894cae74113 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:44.888406 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1299/2566] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:21:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:21:45.406451 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7659c1a2-551d-4123-8a48-074181cdab2b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:45.407954 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7659c1a2-551d-4123-8a48-074181cdab2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:45.471105 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7659c1a2-551d-4123-8a48-074181cdab2b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:45.474695 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3dcc3-dbf7-4c15-9a16-939ac518b45a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:45.476072 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3dcc3-dbf7-4c15-9a16-939ac518b45a service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:45.476355 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3dcc3-dbf7-4c15-9a16-939ac518b45a service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:45.476633 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3dcc3-dbf7-4c15-9a16-939ac518b45a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:45.478452 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-82b3dcc3-dbf7-4c15-9a16-939ac518b45a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:45.546065 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b3dcc3-dbf7-4c15-9a16-939ac518b45a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:45.547580 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b3dcc3-dbf7-4c15-9a16-939ac518b45a service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:45.558685 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1268/2567] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:21:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:21:47.699604 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b664e1e-b488-46d8-90ab-7894cae74113 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:47.713822 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b664e1e-b488-46d8-90ab-7894cae74113 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:47.729268 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829fc7ec-9d85-46f9-8cbe-236476e19be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:47.730425 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829fc7ec-9d85-46f9-8cbe-236476e19be5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:47.730425 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829fc7ec-9d85-46f9-8cbe-236476e19be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:47.730721 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829fc7ec-9d85-46f9-8cbe-236476e19be5 admin admin] PATH_INFO: `/v3/users/4d2dd262ed2649e5b33c0e95cb5811cf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:47.739648 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829fc7ec-9d85-46f9-8cbe-236476e19be5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d2dd262ed2649e5b33c0e95cb5811cf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:47.742598 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829fc7ec-9d85-46f9-8cbe-236476e19be5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:47.857447 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-829fc7ec-9d85-46f9-8cbe-236476e19be5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d2dd262ed2649e5b33c0e95cb5811cf'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:47.883590 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-829fc7ec-9d85-46f9-8cbe-236476e19be5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d2dd262ed2649e5b33c0e95cb5811cf'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:47.884561 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-829fc7ec-9d85-46f9-8cbe-236476e19be5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d2dd262ed2649e5b33c0e95cb5811cf'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:47.900909 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-829fc7ec-9d85-46f9-8cbe-236476e19be5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d2dd262ed2649e5b33c0e95cb5811cf'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:47.902155 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-829fc7ec-9d85-46f9-8cbe-236476e19be5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d2dd262ed2649e5b33c0e95cb5811cf'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:47.955724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1300/2568] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:21:47 2022] DELETE /identity/v3/users/4d2dd262ed2649e5b33c0e95cb5811cf => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:21:47.958301 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6611e5-2ab7-4fbd-8a26-dfd04cbf0137 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:47.959278 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6611e5-2ab7-4fbd-8a26-dfd04cbf0137 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:47.960227 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6611e5-2ab7-4fbd-8a26-dfd04cbf0137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:47.997322 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ee6611e5-2ab7-4fbd-8a26-dfd04cbf0137 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:21:47.998811 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ee6611e5-2ab7-4fbd-8a26-dfd04cbf0137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:48.187863 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1269/2569] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:21:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:21:48.193757 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829fc7ec-9d85-46f9-8cbe-236476e19be5 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:48.195618 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-829fc7ec-9d85-46f9-8cbe-236476e19be5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:48.211590 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:21:48.264627 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829fc7ec-9d85-46f9-8cbe-236476e19be5 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:48.272803 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c22ae-d151-4189-8538-eeb4caeaae5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:48.273847 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c22ae-d151-4189-8538-eeb4caeaae5c service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:48.274373 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c22ae-d151-4189-8538-eeb4caeaae5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:48.275015 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c22ae-d151-4189-8538-eeb4caeaae5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:48.295706 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9c22ae-d151-4189-8538-eeb4caeaae5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:48.297326 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9c22ae-d151-4189-8538-eeb4caeaae5c service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:48.317511 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1301/2570] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:21:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:21:48.857762 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6611e5-2ab7-4fbd-8a26-dfd04cbf0137 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:48.859547 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ee6611e5-2ab7-4fbd-8a26-dfd04cbf0137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:48.875087 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6611e5-2ab7-4fbd-8a26-dfd04cbf0137 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:48.888718 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de93e44-4f98-413c-82d2-3639ff61c4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:48.889623 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de93e44-4f98-413c-82d2-3639ff61c4ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:48.889899 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de93e44-4f98-413c-82d2-3639ff61c4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:48.890190 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de93e44-4f98-413c-82d2-3639ff61c4ab admin admin] PATH_INFO: `/v3/projects/eb3f920d163c4bfa804cb6a1a8f17696` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:48.891623 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de93e44-4f98-413c-82d2-3639ff61c4ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb3f920d163c4bfa804cb6a1a8f17696)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:48.893625 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de93e44-4f98-413c-82d2-3639ff61c4ab admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:48.991373 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:21:49.002695 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9de93e44-4f98-413c-82d2-3639ff61c4ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb3f920d163c4bfa804cb6a1a8f17696', 'request_id': 'req-9de93e44-4f98-413c-82d2-3639ff61c4ab'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:49.019521 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9de93e44-4f98-413c-82d2-3639ff61c4ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb3f920d163c4bfa804cb6a1a8f17696', 'request_id': 'req-9de93e44-4f98-413c-82d2-3639ff61c4ab'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:49.029348 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9de93e44-4f98-413c-82d2-3639ff61c4ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb3f920d163c4bfa804cb6a1a8f17696', 'request_id': 'req-9de93e44-4f98-413c-82d2-3639ff61c4ab'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:49.062276 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9de93e44-4f98-413c-82d2-3639ff61c4ab admin admin] The token cache is being invalidate because project eb3f920d163c4bfa804cb6a1a8f17696 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:21:49.062615 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9de93e44-4f98-413c-82d2-3639ff61c4ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:49.063341 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9de93e44-4f98-413c-82d2-3639ff61c4ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:49.070598 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45ae3c-62e6-47b1-b9e5-263320ae3fcc None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:49.071161 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45ae3c-62e6-47b1-b9e5-263320ae3fcc None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:49.071276 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db45ae3c-62e6-47b1-b9e5-263320ae3fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:49.073418 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1270/2571] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:21:48 2022] DELETE /identity/v3/projects/eb3f920d163c4bfa804cb6a1a8f17696 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:21:49.096318 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-db45ae3c-62e6-47b1-b9e5-263320ae3fcc None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:21:49.097693 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-db45ae3c-62e6-47b1-b9e5-263320ae3fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:49.206208 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1302/2572] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:21:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:21:49.220093 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ebc4e6-66b7-478d-8c5d-9fbc96376b84 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:49.220585 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ebc4e6-66b7-478d-8c5d-9fbc96376b84 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:49.221097 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ebc4e6-66b7-478d-8c5d-9fbc96376b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:49.249784 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-84ebc4e6-66b7-478d-8c5d-9fbc96376b84 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:21:49.251065 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-84ebc4e6-66b7-478d-8c5d-9fbc96376b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:49.265117 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db45ae3c-62e6-47b1-b9e5-263320ae3fcc None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:49.265701 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1271/2573] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:21:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:21:49.277379 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db45ae3c-62e6-47b1-b9e5-263320ae3fcc None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:49.283373 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f335ff0e-f648-4d4d-91f8-f58214d47460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:49.284477 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f335ff0e-f648-4d4d-91f8-f58214d47460 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:49.284919 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f335ff0e-f648-4d4d-91f8-f58214d47460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:49.285465 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f335ff0e-f648-4d4d-91f8-f58214d47460 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:49.286409 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f335ff0e-f648-4d4d-91f8-f58214d47460 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:21:49.287035 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f335ff0e-f648-4d4d-91f8-f58214d47460 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:49.289439 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f335ff0e-f648-4d4d-91f8-f58214d47460 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:49.309269 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1303/2574] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:21:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:21:49.311484 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ebc4e6-66b7-478d-8c5d-9fbc96376b84 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:49.330134 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ebc4e6-66b7-478d-8c5d-9fbc96376b84 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:49.336730 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcca226-1c10-45db-bb5d-249b723038f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:49.337758 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcca226-1c10-45db-bb5d-249b723038f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:49.338199 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcca226-1c10-45db-bb5d-249b723038f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:49.338668 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcca226-1c10-45db-bb5d-249b723038f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:49.339558 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcca226-1c10-45db-bb5d-249b723038f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:49.342122 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcca226-1c10-45db-bb5d-249b723038f2 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:49.389533 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1272/2575] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:21:49 2022] POST /identity/v3/projects => generated 375 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:21:49.391162 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f335ff0e-f648-4d4d-91f8-f58214d47460 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:49.424512 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f335ff0e-f648-4d4d-91f8-f58214d47460 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:49.635685 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd44ab8c-6cc7-4b91-b4b0-1cb2cd6c1f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:49.636450 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd44ab8c-6cc7-4b91-b4b0-1cb2cd6c1f29 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:49.636891 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd44ab8c-6cc7-4b91-b4b0-1cb2cd6c1f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:49.637238 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd44ab8c-6cc7-4b91-b4b0-1cb2cd6c1f29 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:49.638022 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd44ab8c-6cc7-4b91-b4b0-1cb2cd6c1f29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:49.640079 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd44ab8c-6cc7-4b91-b4b0-1cb2cd6c1f29 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:49.674118 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1304/2576] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:21:49 2022] POST /identity/v3/users => generated 288 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:21:49.680212 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcca226-1c10-45db-bb5d-249b723038f2 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:49.692511 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcca226-1c10-45db-bb5d-249b723038f2 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:49.697728 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618377ec-4102-4830-85e7-d4260cd30c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:49.698601 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618377ec-4102-4830-85e7-d4260cd30c20 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:49.699030 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618377ec-4102-4830-85e7-d4260cd30c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:49.699415 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618377ec-4102-4830-85e7-d4260cd30c20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:49.700279 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618377ec-4102-4830-85e7-d4260cd30c20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:49.702099 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618377ec-4102-4830-85e7-d4260cd30c20 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:49.714989 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1273/2577] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:21:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:21:49.721957 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd44ab8c-6cc7-4b91-b4b0-1cb2cd6c1f29 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:49.741682 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd44ab8c-6cc7-4b91-b4b0-1cb2cd6c1f29 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:49.764253 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d875bb0-9013-496f-b85d-5c9140bdff42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:49.764937 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d875bb0-9013-496f-b85d-5c9140bdff42 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:49.765236 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d875bb0-9013-496f-b85d-5c9140bdff42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:49.765539 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d875bb0-9013-496f-b85d-5c9140bdff42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:49.766324 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d875bb0-9013-496f-b85d-5c9140bdff42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:49.768139 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d875bb0-9013-496f-b85d-5c9140bdff42 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:49.794258 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618377ec-4102-4830-85e7-d4260cd30c20 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:49.798019 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1305/2578] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:21:49 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:21:49.811229 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618377ec-4102-4830-85e7-d4260cd30c20 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:49.817081 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0802d8-4623-4a42-b145-83fb7cf5ba33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:49.819805 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0802d8-4623-4a42-b145-83fb7cf5ba33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:49.820280 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0802d8-4623-4a42-b145-83fb7cf5ba33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:49.820857 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0802d8-4623-4a42-b145-83fb7cf5ba33 admin admin] PATH_INFO: `/v3/projects/9c3ebf345dbe46f7a137e1a62dee5f33/users/4cb25251cbd442a89f0a73bf35e76c3c/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:49.844613 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0802d8-4623-4a42-b145-83fb7cf5ba33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c3ebf345dbe46f7a137e1a62dee5f33, user_id=4cb25251cbd442a89f0a73bf35e76c3c, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:49.847920 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0802d8-4623-4a42-b145-83fb7cf5ba33 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:49.886969 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1274/2579] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:21:49 2022] PUT /identity/v3/projects/9c3ebf345dbe46f7a137e1a62dee5f33/users/4cb25251cbd442a89f0a73bf35e76c3c/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:21:49.888689 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250e619-4c1a-4693-a3b5-ca6effe9c4f5 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:49.888973 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250e619-4c1a-4693-a3b5-ca6effe9c4f5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:49.889256 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250e619-4c1a-4693-a3b5-ca6effe9c4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:50.014064 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7250e619-4c1a-4693-a3b5-ca6effe9c4f5 None None] MFA Rules not processed for user `4cb25251cbd442a89f0a73bf35e76c3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:21:50.015374 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7250e619-4c1a-4693-a3b5-ca6effe9c4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:50.044717 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:21:50.189771 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1306/2580] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:21:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:21:50.202477 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0802d8-4623-4a42-b145-83fb7cf5ba33 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:50.207907 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-8f0802d8-4623-4a42-b145-83fb7cf5ba33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:50.212954 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250e619-4c1a-4693-a3b5-ca6effe9c4f5 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:50.214413 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7250e619-4c1a-4693-a3b5-ca6effe9c4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:50.282715 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0802d8-4623-4a42-b145-83fb7cf5ba33 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:50.286364 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127ec64-b966-46df-9c90-34060561eb0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:50.287043 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127ec64-b966-46df-9c90-34060561eb0c service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:50.287365 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127ec64-b966-46df-9c90-34060561eb0c service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:50.287711 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127ec64-b966-46df-9c90-34060561eb0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:50.292361 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250e619-4c1a-4693-a3b5-ca6effe9c4f5 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:50.296659 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5127ec64-b966-46df-9c90-34060561eb0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:50.297015 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eeaa15-64b4-40d5-a03d-273f8dc57492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:50.297942 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eeaa15-64b4-40d5-a03d-273f8dc57492 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:50.298088 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5127ec64-b966-46df-9c90-34060561eb0c service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:50.298232 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eeaa15-64b4-40d5-a03d-273f8dc57492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:50.298508 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eeaa15-64b4-40d5-a03d-273f8dc57492 admin admin] PATH_INFO: `/v3/users/387048f42b634693a7e827a8d88ef89b` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:50.299737 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eeaa15-64b4-40d5-a03d-273f8dc57492 admin admin] RBAC: Authorizing `identity:delete_user(user_id=387048f42b634693a7e827a8d88ef89b)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:50.301676 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eeaa15-64b4-40d5-a03d-273f8dc57492 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:50.307502 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1275/2581] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:21:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:21:50.368034 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b8eeaa15-64b4-40d5-a03d-273f8dc57492 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '387048f42b634693a7e827a8d88ef89b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:50.375497 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b8eeaa15-64b4-40d5-a03d-273f8dc57492 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '387048f42b634693a7e827a8d88ef89b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:50.376234 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b8eeaa15-64b4-40d5-a03d-273f8dc57492 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '387048f42b634693a7e827a8d88ef89b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:50.385296 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b8eeaa15-64b4-40d5-a03d-273f8dc57492 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '387048f42b634693a7e827a8d88ef89b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:50.386146 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b8eeaa15-64b4-40d5-a03d-273f8dc57492 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '387048f42b634693a7e827a8d88ef89b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:50.413463 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1307/2582] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:21:50 2022] DELETE /identity/v3/users/387048f42b634693a7e827a8d88ef89b => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:21:50.620625 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127ec64-b966-46df-9c90-34060561eb0c service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:50.650572 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127ec64-b966-46df-9c90-34060561eb0c service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:50.741102 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bb9cfa-0225-4f05-b452-15f304e60045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:50.742441 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb9cfa-0225-4f05-b452-15f304e60045 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:50.742835 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb9cfa-0225-4f05-b452-15f304e60045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:50.743251 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bb9cfa-0225-4f05-b452-15f304e60045 admin admin] PATH_INFO: `/v3/projects/0bf15a964f8048b48bc794051cd56add` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:50.744652 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bb9cfa-0225-4f05-b452-15f304e60045 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bf15a964f8048b48bc794051cd56add)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:50.746722 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bb9cfa-0225-4f05-b452-15f304e60045 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:50.766816 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:21:50.812727 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-95bb9cfa-0225-4f05-b452-15f304e60045 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bf15a964f8048b48bc794051cd56add', 'request_id': 'req-95bb9cfa-0225-4f05-b452-15f304e60045'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:50.820292 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-95bb9cfa-0225-4f05-b452-15f304e60045 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bf15a964f8048b48bc794051cd56add', 'request_id': 'req-95bb9cfa-0225-4f05-b452-15f304e60045'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:50.834877 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-95bb9cfa-0225-4f05-b452-15f304e60045 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bf15a964f8048b48bc794051cd56add', 'request_id': 'req-95bb9cfa-0225-4f05-b452-15f304e60045'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:50.851992 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-95bb9cfa-0225-4f05-b452-15f304e60045 admin admin] The token cache is being invalidate because project 0bf15a964f8048b48bc794051cd56add was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:21:50.852286 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-95bb9cfa-0225-4f05-b452-15f304e60045 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:50.852949 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-95bb9cfa-0225-4f05-b452-15f304e60045 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:50.854915 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1276/2583] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:21:50 2022] DELETE /identity/v3/projects/0bf15a964f8048b48bc794051cd56add => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:21:50.860567 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a9cce-dfce-4152-b09a-07122711b0f2 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:50.861095 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a9cce-dfce-4152-b09a-07122711b0f2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:50.861571 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33a9cce-dfce-4152-b09a-07122711b0f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:50.889893 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b33a9cce-dfce-4152-b09a-07122711b0f2 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:21:50.891177 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b33a9cce-dfce-4152-b09a-07122711b0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:50.931815 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:21:50.982532 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1308/2584] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:21:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:21:50.997852 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da194a0e-13ec-406a-b545-b0a71752a997 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:50.998377 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da194a0e-13ec-406a-b545-b0a71752a997 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:50.998645 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da194a0e-13ec-406a-b545-b0a71752a997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:51.027314 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-da194a0e-13ec-406a-b545-b0a71752a997 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:21:51.028837 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-da194a0e-13ec-406a-b545-b0a71752a997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:51.044220 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1277/2585] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:21:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:21:51.048871 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a9cce-dfce-4152-b09a-07122711b0f2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:51.060521 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33a9cce-dfce-4152-b09a-07122711b0f2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:51.070889 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fea288-5ab6-4fde-a967-2c12df6f3063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:51.072013 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fea288-5ab6-4fde-a967-2c12df6f3063 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:51.072618 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fea288-5ab6-4fde-a967-2c12df6f3063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:51.073068 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fea288-5ab6-4fde-a967-2c12df6f3063 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:51.073980 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fea288-5ab6-4fde-a967-2c12df6f3063 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:21:51.074456 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fea288-5ab6-4fde-a967-2c12df6f3063 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:51.076278 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fea288-5ab6-4fde-a967-2c12df6f3063 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:51.095075 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da194a0e-13ec-406a-b545-b0a71752a997 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:51.100036 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1309/2586] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:21:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:21:51.105177 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da194a0e-13ec-406a-b545-b0a71752a997 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:51.110258 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53790050-1a37-49a1-b4a3-fd89407b067d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:51.110980 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53790050-1a37-49a1-b4a3-fd89407b067d admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:51.111265 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53790050-1a37-49a1-b4a3-fd89407b067d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:51.111546 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53790050-1a37-49a1-b4a3-fd89407b067d admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:51.112888 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53790050-1a37-49a1-b4a3-fd89407b067d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:51.114888 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53790050-1a37-49a1-b4a3-fd89407b067d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:51.162943 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1278/2587] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:21:51 2022] POST /identity/v3/projects => generated 405 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:21:51.163270 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fea288-5ab6-4fde-a967-2c12df6f3063 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:51.186168 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fea288-5ab6-4fde-a967-2c12df6f3063 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:51.253537 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311bb0ca-7db8-47f3-a8b8-5cbd0987137e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:51.255066 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311bb0ca-7db8-47f3-a8b8-5cbd0987137e admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:51.255570 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311bb0ca-7db8-47f3-a8b8-5cbd0987137e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:51.255844 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311bb0ca-7db8-47f3-a8b8-5cbd0987137e admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:51.257174 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311bb0ca-7db8-47f3-a8b8-5cbd0987137e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:51.259116 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311bb0ca-7db8-47f3-a8b8-5cbd0987137e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:51.285463 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1310/2588] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:21:51 2022] POST /identity/v3/users => generated 303 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:21:51.290129 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53790050-1a37-49a1-b4a3-fd89407b067d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:51.300585 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53790050-1a37-49a1-b4a3-fd89407b067d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:51.306416 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3635995-f843-4926-a64b-8077db983a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:51.307120 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3635995-f843-4926-a64b-8077db983a08 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:51.307534 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3635995-f843-4926-a64b-8077db983a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:51.307818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3635995-f843-4926-a64b-8077db983a08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:51.308546 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3635995-f843-4926-a64b-8077db983a08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:51.310153 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3635995-f843-4926-a64b-8077db983a08 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:51.322293 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1279/2589] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:21:51.326222 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311bb0ca-7db8-47f3-a8b8-5cbd0987137e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:51.336744 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311bb0ca-7db8-47f3-a8b8-5cbd0987137e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:51.341387 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0c8e49-0a03-425c-b513-5272399f2b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:51.342058 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c8e49-0a03-425c-b513-5272399f2b6f admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:51.342341 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c8e49-0a03-425c-b513-5272399f2b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:51.342614 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0c8e49-0a03-425c-b513-5272399f2b6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:51.343371 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0c8e49-0a03-425c-b513-5272399f2b6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:51.345199 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0c8e49-0a03-425c-b513-5272399f2b6f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:51.355244 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1311/2590] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:21:51.359567 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3635995-f843-4926-a64b-8077db983a08 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:51.370084 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3635995-f843-4926-a64b-8077db983a08 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:51.374804 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c13255-342f-4637-90a8-71ce74a7430d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:51.375720 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c13255-342f-4637-90a8-71ce74a7430d admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:51.376000 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c13255-342f-4637-90a8-71ce74a7430d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:51.376277 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c13255-342f-4637-90a8-71ce74a7430d admin admin] PATH_INFO: `/v3/projects/a919e1e04f5f44cab53472a0367c9501/users/e8d34bbdb86a4153835695b29caa1c8d/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:51.392378 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c13255-342f-4637-90a8-71ce74a7430d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a919e1e04f5f44cab53472a0367c9501, user_id=e8d34bbdb86a4153835695b29caa1c8d, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:51.395053 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c13255-342f-4637-90a8-71ce74a7430d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:51.409127 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1280/2591] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:21:51 2022] PUT /identity/v3/projects/a919e1e04f5f44cab53472a0367c9501/users/e8d34bbdb86a4153835695b29caa1c8d/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:21:51.420277 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f6db00-c32f-4c78-9729-df3a560d0481 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:51.420571 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f6db00-c32f-4c78-9729-df3a560d0481 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:51.420858 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f6db00-c32f-4c78-9729-df3a560d0481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:51.451232 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b4f6db00-c32f-4c78-9729-df3a560d0481 None None] MFA Rules not processed for user `e8d34bbdb86a4153835695b29caa1c8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:21:51.452524 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b4f6db00-c32f-4c78-9729-df3a560d0481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:51.553970 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1312/2592] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:21:51 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:21:51.558774 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c13255-342f-4637-90a8-71ce74a7430d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:51.661305 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c13255-342f-4637-90a8-71ce74a7430d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:51.667186 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f66332-e7c2-4634-9db8-3ac23d2427fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:51.668037 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f66332-e7c2-4634-9db8-3ac23d2427fb admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:51.668425 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f66332-e7c2-4634-9db8-3ac23d2427fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:51.668811 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f66332-e7c2-4634-9db8-3ac23d2427fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:51.669610 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f66332-e7c2-4634-9db8-3ac23d2427fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:51.671866 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f66332-e7c2-4634-9db8-3ac23d2427fb admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:51.715611 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1281/2593] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:21:51 2022] POST /identity/v3/projects => generated 403 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:21:51.715844 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f6db00-c32f-4c78-9729-df3a560d0481 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:51.717750 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316fd5d-21a4-4b09-a3d3-4a7a609c5a73 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:51.718111 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316fd5d-21a4-4b09-a3d3-4a7a609c5a73 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:51.718652 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b316fd5d-21a4-4b09-a3d3-4a7a609c5a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:51.735569 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f6db00-c32f-4c78-9729-df3a560d0481 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:51.739390 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b316fd5d-21a4-4b09-a3d3-4a7a609c5a73 None None] MFA Rules not processed for user `4cb25251cbd442a89f0a73bf35e76c3c`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:21:51.741600 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b316fd5d-21a4-4b09-a3d3-4a7a609c5a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:51.763043 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:21:51.766700 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:21:51.822581 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55367e65-7879-4b28-a85b-7924e9e81d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:51.823302 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55367e65-7879-4b28-a85b-7924e9e81d07 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:51.823733 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55367e65-7879-4b28-a85b-7924e9e81d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:51.824013 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55367e65-7879-4b28-a85b-7924e9e81d07 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:51.824711 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55367e65-7879-4b28-a85b-7924e9e81d07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:51.826660 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55367e65-7879-4b28-a85b-7924e9e81d07 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:51.855665 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1282/2594] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:21:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:21:51.860736 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1313/2595] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:21:51 2022] POST /identity/v3/users => generated 308 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:21:51.864533 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316fd5d-21a4-4b09-a3d3-4a7a609c5a73 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:51.881511 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b316fd5d-21a4-4b09-a3d3-4a7a609c5a73 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:51.886491 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c1d321-d20c-4bb3-9e1f-5b92e69eeba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:51.887286 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1d321-d20c-4bb3-9e1f-5b92e69eeba8 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:51.887648 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1d321-d20c-4bb3-9e1f-5b92e69eeba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:51.888268 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1d321-d20c-4bb3-9e1f-5b92e69eeba8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:51.889131 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c1d321-d20c-4bb3-9e1f-5b92e69eeba8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:51.890822 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c1d321-d20c-4bb3-9e1f-5b92e69eeba8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:51.902492 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1283/2596] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:21:51.906689 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55367e65-7879-4b28-a85b-7924e9e81d07 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:51.917960 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55367e65-7879-4b28-a85b-7924e9e81d07 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:51.925637 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edcfc0c-f3ba-4781-af78-7de975c5c0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:51.926757 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edcfc0c-f3ba-4781-af78-7de975c5c0a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:51.927243 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edcfc0c-f3ba-4781-af78-7de975c5c0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:51.927688 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edcfc0c-f3ba-4781-af78-7de975c5c0a4 admin admin] PATH_INFO: `/v3/projects/98dcdc10448041448ccb4c04471f1ff7/users/d1d21be7d8594117b1121246ee360b0b/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:51.953534 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edcfc0c-f3ba-4781-af78-7de975c5c0a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98dcdc10448041448ccb4c04471f1ff7, user_id=d1d21be7d8594117b1121246ee360b0b, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:51.957082 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edcfc0c-f3ba-4781-af78-7de975c5c0a4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:51.974383 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1314/2597] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:21:51 2022] PUT /identity/v3/projects/98dcdc10448041448ccb4c04471f1ff7/users/d1d21be7d8594117b1121246ee360b0b/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:21:51.978649 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c1d321-d20c-4bb3-9e1f-5b92e69eeba8 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:51.998285 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c1d321-d20c-4bb3-9e1f-5b92e69eeba8 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:52.096505 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a1c5db-5f8c-4c2f-b154-4c10f1a3ef9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:52.097501 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1c5db-5f8c-4c2f-b154-4c10f1a3ef9b admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:52.097790 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1c5db-5f8c-4c2f-b154-4c10f1a3ef9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:52.098071 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1c5db-5f8c-4c2f-b154-4c10f1a3ef9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:52.098876 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a1c5db-5f8c-4c2f-b154-4c10f1a3ef9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:52.101396 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a1c5db-5f8c-4c2f-b154-4c10f1a3ef9b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:52.115546 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1284/2598] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:21:52.119515 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edcfc0c-f3ba-4781-af78-7de975c5c0a4 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:52.134462 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edcfc0c-f3ba-4781-af78-7de975c5c0a4 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:52.140781 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca38043-c84d-402a-b613-5c2c7f5b1702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:52.141982 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca38043-c84d-402a-b613-5c2c7f5b1702 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:52.142492 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca38043-c84d-402a-b613-5c2c7f5b1702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:52.143004 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca38043-c84d-402a-b613-5c2c7f5b1702 admin admin] PATH_INFO: `/v3/projects/98dcdc10448041448ccb4c04471f1ff7/users/d1d21be7d8594117b1121246ee360b0b/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:52.146183 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca38043-c84d-402a-b613-5c2c7f5b1702 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98dcdc10448041448ccb4c04471f1ff7, user_id=d1d21be7d8594117b1121246ee360b0b, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:52.149807 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca38043-c84d-402a-b613-5c2c7f5b1702 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:52.194789 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1315/2599] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:21:52 2022] PUT /identity/v3/projects/98dcdc10448041448ccb4c04471f1ff7/users/d1d21be7d8594117b1121246ee360b0b/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:21:52.205061 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef810a3a-3c5d-4aa6-b6eb-aac846277cc1 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:52.205991 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef810a3a-3c5d-4aa6-b6eb-aac846277cc1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:52.206701 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef810a3a-3c5d-4aa6-b6eb-aac846277cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:52.247609 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ef810a3a-3c5d-4aa6-b6eb-aac846277cc1 None None] MFA Rules not processed for user `d1d21be7d8594117b1121246ee360b0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:21:52.248831 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ef810a3a-3c5d-4aa6-b6eb-aac846277cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:52.385564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1285/2600] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:21:52 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:21:52.407985 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca38043-c84d-402a-b613-5c2c7f5b1702 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:52.409786 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4ca38043-c84d-402a-b613-5c2c7f5b1702 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:52.530784 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca38043-c84d-402a-b613-5c2c7f5b1702 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:52.539589 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626710ad-b36f-4ac1-a33c-53b4d45f6722 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:52.540481 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626710ad-b36f-4ac1-a33c-53b4d45f6722 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:52.540925 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626710ad-b36f-4ac1-a33c-53b4d45f6722 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:52.541359 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626710ad-b36f-4ac1-a33c-53b4d45f6722 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:52.557382 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626710ad-b36f-4ac1-a33c-53b4d45f6722 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:52.559107 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626710ad-b36f-4ac1-a33c-53b4d45f6722 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:52.570761 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1316/2601] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:21:52 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:21:52.785955 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef810a3a-3c5d-4aa6-b6eb-aac846277cc1 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:52.842684 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:21:52.869347 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef810a3a-3c5d-4aa6-b6eb-aac846277cc1 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:52.874102 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f47c00-aeed-4e29-b80a-32409877bcc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:52.875092 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f47c00-aeed-4e29-b80a-32409877bcc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:52.875376 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f47c00-aeed-4e29-b80a-32409877bcc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:52.875656 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f47c00-aeed-4e29-b80a-32409877bcc7 admin admin] PATH_INFO: `/v3/users/e8d34bbdb86a4153835695b29caa1c8d` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:52.876870 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f47c00-aeed-4e29-b80a-32409877bcc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8d34bbdb86a4153835695b29caa1c8d)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:52.878830 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f47c00-aeed-4e29-b80a-32409877bcc7 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:52.953772 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-55f47c00-aeed-4e29-b80a-32409877bcc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8d34bbdb86a4153835695b29caa1c8d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:52.963857 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-55f47c00-aeed-4e29-b80a-32409877bcc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8d34bbdb86a4153835695b29caa1c8d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:52.973494 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-55f47c00-aeed-4e29-b80a-32409877bcc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8d34bbdb86a4153835695b29caa1c8d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:52.974197 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-55f47c00-aeed-4e29-b80a-32409877bcc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8d34bbdb86a4153835695b29caa1c8d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:52.975611 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-55f47c00-aeed-4e29-b80a-32409877bcc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8d34bbdb86a4153835695b29caa1c8d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:52.997945 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1286/2602] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:21:52 2022] DELETE /identity/v3/users/e8d34bbdb86a4153835695b29caa1c8d => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:21:53.002201 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64190f00-458a-46e5-a7cb-34f81ce44e0b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:53.002490 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64190f00-458a-46e5-a7cb-34f81ce44e0b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:53.002769 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64190f00-458a-46e5-a7cb-34f81ce44e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:53.028103 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-64190f00-458a-46e5-a7cb-34f81ce44e0b None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:21:53.030012 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-64190f00-458a-46e5-a7cb-34f81ce44e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:53.125604 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1317/2603] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:21:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:21:53.132277 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f47c00-aeed-4e29-b80a-32409877bcc7 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:53.133912 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-55f47c00-aeed-4e29-b80a-32409877bcc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:53.218073 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f47c00-aeed-4e29-b80a-32409877bcc7 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:53.231922 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e389b-7fee-4d1b-8c49-4c90bcbcb093 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:53.232963 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e389b-7fee-4d1b-8c49-4c90bcbcb093 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:53.233435 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e389b-7fee-4d1b-8c49-4c90bcbcb093 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:53.233791 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e389b-7fee-4d1b-8c49-4c90bcbcb093 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:53.243680 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244e389b-7fee-4d1b-8c49-4c90bcbcb093 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:53.245499 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244e389b-7fee-4d1b-8c49-4c90bcbcb093 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:53.256939 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1287/2604] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:21:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:21:53.612192 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64190f00-458a-46e5-a7cb-34f81ce44e0b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:53.614116 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-64190f00-458a-46e5-a7cb-34f81ce44e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:53.628227 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64190f00-458a-46e5-a7cb-34f81ce44e0b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:53.636232 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b5035-f03d-464e-935c-fecebec9e8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:53.662956 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b5035-f03d-464e-935c-fecebec9e8fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:53.662956 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b5035-f03d-464e-935c-fecebec9e8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:53.662956 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b5035-f03d-464e-935c-fecebec9e8fd admin admin] PATH_INFO: `/v3/projects/a919e1e04f5f44cab53472a0367c9501` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:53.662956 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409b5035-f03d-464e-935c-fecebec9e8fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=a919e1e04f5f44cab53472a0367c9501)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:53.662956 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409b5035-f03d-464e-935c-fecebec9e8fd admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:53.716130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:21:53.766768 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-409b5035-f03d-464e-935c-fecebec9e8fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a919e1e04f5f44cab53472a0367c9501', 'request_id': 'req-409b5035-f03d-464e-935c-fecebec9e8fd'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:53.783754 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-409b5035-f03d-464e-935c-fecebec9e8fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a919e1e04f5f44cab53472a0367c9501', 'request_id': 'req-409b5035-f03d-464e-935c-fecebec9e8fd'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:53.807913 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-409b5035-f03d-464e-935c-fecebec9e8fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a919e1e04f5f44cab53472a0367c9501', 'request_id': 'req-409b5035-f03d-464e-935c-fecebec9e8fd'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:53.824667 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-409b5035-f03d-464e-935c-fecebec9e8fd admin admin] The token cache is being invalidate because project a919e1e04f5f44cab53472a0367c9501 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:21:53.825380 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-409b5035-f03d-464e-935c-fecebec9e8fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:53.827022 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-409b5035-f03d-464e-935c-fecebec9e8fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:53.829845 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1318/2605] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:21:53 2022] DELETE /identity/v3/projects/a919e1e04f5f44cab53472a0367c9501 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:21:53.834964 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e389b-7fee-4d1b-8c49-4c90bcbcb093 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:53.847494 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e389b-7fee-4d1b-8c49-4c90bcbcb093 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:53.919312 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e184cf81-4363-428f-8b87-48b7a81cd156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:53.920359 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e184cf81-4363-428f-8b87-48b7a81cd156 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:53.920692 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e184cf81-4363-428f-8b87-48b7a81cd156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:53.920972 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e184cf81-4363-428f-8b87-48b7a81cd156 admin admin] PATH_INFO: `/v3/users/d1d21be7d8594117b1121246ee360b0b` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:53.922257 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e184cf81-4363-428f-8b87-48b7a81cd156 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1d21be7d8594117b1121246ee360b0b)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:53.924497 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e184cf81-4363-428f-8b87-48b7a81cd156 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:53.927747 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:21:53.995785 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e184cf81-4363-428f-8b87-48b7a81cd156 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1d21be7d8594117b1121246ee360b0b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:54.004512 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e184cf81-4363-428f-8b87-48b7a81cd156 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1d21be7d8594117b1121246ee360b0b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:54.019305 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e184cf81-4363-428f-8b87-48b7a81cd156 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1d21be7d8594117b1121246ee360b0b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:54.019305 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e184cf81-4363-428f-8b87-48b7a81cd156 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1d21be7d8594117b1121246ee360b0b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:54.019305 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e184cf81-4363-428f-8b87-48b7a81cd156 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1d21be7d8594117b1121246ee360b0b'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:54.042145 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1288/2606] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:21:53 2022] DELETE /identity/v3/users/d1d21be7d8594117b1121246ee360b0b => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:21:54.349977 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b5035-f03d-464e-935c-fecebec9e8fd admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:54.351880 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-409b5035-f03d-464e-935c-fecebec9e8fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:54.364975 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b5035-f03d-464e-935c-fecebec9e8fd admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:54.429637 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01906bb-5e6c-4296-8e51-ac4a61b4f207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:54.430809 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01906bb-5e6c-4296-8e51-ac4a61b4f207 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:54.431127 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01906bb-5e6c-4296-8e51-ac4a61b4f207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:54.431408 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01906bb-5e6c-4296-8e51-ac4a61b4f207 admin admin] PATH_INFO: `/v3/projects/98dcdc10448041448ccb4c04471f1ff7` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:54.432600 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01906bb-5e6c-4296-8e51-ac4a61b4f207 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98dcdc10448041448ccb4c04471f1ff7)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:54.434507 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01906bb-5e6c-4296-8e51-ac4a61b4f207 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:54.482708 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e01906bb-5e6c-4296-8e51-ac4a61b4f207 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98dcdc10448041448ccb4c04471f1ff7', 'request_id': 'req-e01906bb-5e6c-4296-8e51-ac4a61b4f207'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:54.490321 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e01906bb-5e6c-4296-8e51-ac4a61b4f207 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98dcdc10448041448ccb4c04471f1ff7', 'request_id': 'req-e01906bb-5e6c-4296-8e51-ac4a61b4f207'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:54.495811 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e01906bb-5e6c-4296-8e51-ac4a61b4f207 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98dcdc10448041448ccb4c04471f1ff7', 'request_id': 'req-e01906bb-5e6c-4296-8e51-ac4a61b4f207'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:54.506636 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e01906bb-5e6c-4296-8e51-ac4a61b4f207 admin admin] The token cache is being invalidate because project 98dcdc10448041448ccb4c04471f1ff7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:21:54.506953 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e01906bb-5e6c-4296-8e51-ac4a61b4f207 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:54.507589 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e01906bb-5e6c-4296-8e51-ac4a61b4f207 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:21:54.509124 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1319/2607] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:21:54 2022] DELETE /identity/v3/projects/98dcdc10448041448ccb4c04471f1ff7 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:21:54.514009 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3161abe0-33df-45a3-8aab-b591e0f81888 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:54.514886 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3161abe0-33df-45a3-8aab-b591e0f81888 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:54.514886 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3161abe0-33df-45a3-8aab-b591e0f81888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:54.533710 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3161abe0-33df-45a3-8aab-b591e0f81888 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:21:54.534951 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3161abe0-33df-45a3-8aab-b591e0f81888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:54.593257 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1289/2608] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:21:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:21:54.601950 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e9d1b-cf51-4e4f-8030-2b78bf962783 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:54.602236 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e9d1b-cf51-4e4f-8030-2b78bf962783 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:54.602516 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75e9d1b-cf51-4e4f-8030-2b78bf962783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:54.620532 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d75e9d1b-cf51-4e4f-8030-2b78bf962783 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:21:54.621715 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d75e9d1b-cf51-4e4f-8030-2b78bf962783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:54.630155 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1320/2609] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:21:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:21:54.634482 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3161abe0-33df-45a3-8aab-b591e0f81888 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:54.644181 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3161abe0-33df-45a3-8aab-b591e0f81888 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:54.649316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbdfeae-2384-4d82-811b-bb68de939be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:54.649960 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbdfeae-2384-4d82-811b-bb68de939be5 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:54.650241 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbdfeae-2384-4d82-811b-bb68de939be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:54.650517 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbdfeae-2384-4d82-811b-bb68de939be5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:54.651279 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbdfeae-2384-4d82-811b-bb68de939be5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:21:54.651583 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbdfeae-2384-4d82-811b-bb68de939be5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:54.653204 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbdfeae-2384-4d82-811b-bb68de939be5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:54.665297 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1290/2610] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:21:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:21:54.669571 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e9d1b-cf51-4e4f-8030-2b78bf962783 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:54.678966 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75e9d1b-cf51-4e4f-8030-2b78bf962783 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:54.683587 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60dbca9-2f35-4c8d-85f8-c5f29dc5fa4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:54.684234 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60dbca9-2f35-4c8d-85f8-c5f29dc5fa4a admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:54.684514 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60dbca9-2f35-4c8d-85f8-c5f29dc5fa4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:54.684786 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60dbca9-2f35-4c8d-85f8-c5f29dc5fa4a admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:54.685466 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60dbca9-2f35-4c8d-85f8-c5f29dc5fa4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:54.687377 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60dbca9-2f35-4c8d-85f8-c5f29dc5fa4a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:54.716805 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1321/2611] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:21:54 2022] POST /identity/v3/projects => generated 357 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:21:54.718093 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbdfeae-2384-4d82-811b-bb68de939be5 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:54.728757 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbdfeae-2384-4d82-811b-bb68de939be5 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:54.786240 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d3166-4041-4b9c-8d3d-29f1e67d53f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:54.786840 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d3166-4041-4b9c-8d3d-29f1e67d53f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:54.787241 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d3166-4041-4b9c-8d3d-29f1e67d53f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:54.787633 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d3166-4041-4b9c-8d3d-29f1e67d53f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:54.788368 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5d3166-4041-4b9c-8d3d-29f1e67d53f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:54.790297 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5d3166-4041-4b9c-8d3d-29f1e67d53f3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:54.807852 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:21:54.815048 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1291/2612] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:21:54 2022] POST /identity/v3/users => generated 279 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:21:54.819738 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60dbca9-2f35-4c8d-85f8-c5f29dc5fa4a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:54.826768 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:21:54.830766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60dbca9-2f35-4c8d-85f8-c5f29dc5fa4a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:54.838325 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f9f12-961e-47a8-9598-fd42e8a5e3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:54.839118 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f9f12-961e-47a8-9598-fd42e8a5e3b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:54.839867 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f9f12-961e-47a8-9598-fd42e8a5e3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:54.840295 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f9f12-961e-47a8-9598-fd42e8a5e3b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:54.842896 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f9f12-961e-47a8-9598-fd42e8a5e3b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:54.843135 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f9f12-961e-47a8-9598-fd42e8a5e3b0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:54.855074 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1322/2613] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:21:54 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:21:54.859150 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d3166-4041-4b9c-8d3d-29f1e67d53f3 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:54.869175 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d3166-4041-4b9c-8d3d-29f1e67d53f3 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:54.873760 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acabeda7-e706-462f-901f-5794e7d7b455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:54.874416 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acabeda7-e706-462f-901f-5794e7d7b455 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:54.874696 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acabeda7-e706-462f-901f-5794e7d7b455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:54.875007 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acabeda7-e706-462f-901f-5794e7d7b455 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:54.875720 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acabeda7-e706-462f-901f-5794e7d7b455 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:54.877311 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acabeda7-e706-462f-901f-5794e7d7b455 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:54.894377 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1292/2614] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:21:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:21:54.895864 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f9f12-961e-47a8-9598-fd42e8a5e3b0 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:54.906442 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f9f12-961e-47a8-9598-fd42e8a5e3b0 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:54.911168 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b264f45-5e32-4fab-a62e-26a33bfdde7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:54.911998 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b264f45-5e32-4fab-a62e-26a33bfdde7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:54.912318 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b264f45-5e32-4fab-a62e-26a33bfdde7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:54.912582 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b264f45-5e32-4fab-a62e-26a33bfdde7a admin admin] PATH_INFO: `/v3/projects/76222af4f3af4f24bfec86bc29ddd65c/users/ac93fff21b8a41faaad7008d354a3296/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:54.928731 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b264f45-5e32-4fab-a62e-26a33bfdde7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=76222af4f3af4f24bfec86bc29ddd65c, user_id=ac93fff21b8a41faaad7008d354a3296, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:54.931413 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b264f45-5e32-4fab-a62e-26a33bfdde7a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:54.944916 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1323/2615] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:21:54 2022] PUT /identity/v3/projects/76222af4f3af4f24bfec86bc29ddd65c/users/ac93fff21b8a41faaad7008d354a3296/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:21:54.956326 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e0d038-7f13-4d4f-b1c5-e009967fd889 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:54.956710 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e0d038-7f13-4d4f-b1c5-e009967fd889 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:54.956993 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e0d038-7f13-4d4f-b1c5-e009967fd889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:54.985389 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-09e0d038-7f13-4d4f-b1c5-e009967fd889 None None] MFA Rules not processed for user `ac93fff21b8a41faaad7008d354a3296`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:21:54.986708 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-09e0d038-7f13-4d4f-b1c5-e009967fd889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:55.053851 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1293/2616] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:21:54 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:21:55.057974 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b264f45-5e32-4fab-a62e-26a33bfdde7a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:55.152444 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b264f45-5e32-4fab-a62e-26a33bfdde7a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:55.158208 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3642f1-2137-4d5d-ad08-9aa11582c402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:55.158931 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3642f1-2137-4d5d-ad08-9aa11582c402 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:55.159224 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3642f1-2137-4d5d-ad08-9aa11582c402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:55.159771 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3642f1-2137-4d5d-ad08-9aa11582c402 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:55.161089 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3642f1-2137-4d5d-ad08-9aa11582c402 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:55.162808 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3642f1-2137-4d5d-ad08-9aa11582c402 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:55.194890 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1324/2617] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:21:55 2022] POST /identity/v3/projects => generated 357 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:21:55.197404 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e0d038-7f13-4d4f-b1c5-e009967fd889 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:55.208385 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e0d038-7f13-4d4f-b1c5-e009967fd889 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:55.270892 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc987b3-aa63-4209-ab75-2a7d87ae2b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:55.271645 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc987b3-aa63-4209-ab75-2a7d87ae2b97 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:55.272010 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc987b3-aa63-4209-ab75-2a7d87ae2b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:55.272289 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc987b3-aa63-4209-ab75-2a7d87ae2b97 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:55.272985 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc987b3-aa63-4209-ab75-2a7d87ae2b97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:55.275105 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc987b3-aa63-4209-ab75-2a7d87ae2b97 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:55.300808 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1294/2618] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:21:55 2022] POST /identity/v3/users => generated 285 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:21:55.308126 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3642f1-2137-4d5d-ad08-9aa11582c402 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:55.320190 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3642f1-2137-4d5d-ad08-9aa11582c402 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:55.325405 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89001ec4-545c-414b-b57b-00069570ecb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:55.326160 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89001ec4-545c-414b-b57b-00069570ecb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:55.326447 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89001ec4-545c-414b-b57b-00069570ecb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:55.326729 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89001ec4-545c-414b-b57b-00069570ecb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:55.327503 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89001ec4-545c-414b-b57b-00069570ecb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:55.329097 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89001ec4-545c-414b-b57b-00069570ecb9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:55.340785 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1325/2619] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:21:55.345340 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc987b3-aa63-4209-ab75-2a7d87ae2b97 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:55.356601 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc987b3-aa63-4209-ab75-2a7d87ae2b97 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:55.361519 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5af476-ffe7-43a0-ab84-9e0f3e369c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:55.362404 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5af476-ffe7-43a0-ab84-9e0f3e369c29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:55.362691 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5af476-ffe7-43a0-ab84-9e0f3e369c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:55.363008 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5af476-ffe7-43a0-ab84-9e0f3e369c29 admin admin] PATH_INFO: `/v3/projects/336ae9e3da4142c794b33bd29a738c6b/users/4119fd46334647cfa262febd43acb3bd/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:55.378381 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5af476-ffe7-43a0-ab84-9e0f3e369c29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=336ae9e3da4142c794b33bd29a738c6b, user_id=4119fd46334647cfa262febd43acb3bd, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:55.381211 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5af476-ffe7-43a0-ab84-9e0f3e369c29 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:55.396054 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1295/2620] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:21:55 2022] PUT /identity/v3/projects/336ae9e3da4142c794b33bd29a738c6b/users/4119fd46334647cfa262febd43acb3bd/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:21:55.400842 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89001ec4-545c-414b-b57b-00069570ecb9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:55.410780 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89001ec4-545c-414b-b57b-00069570ecb9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:55.469885 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720986fa-2432-4780-a9f7-0a3e758080c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:55.470553 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720986fa-2432-4780-a9f7-0a3e758080c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:55.470838 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720986fa-2432-4780-a9f7-0a3e758080c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:55.471146 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720986fa-2432-4780-a9f7-0a3e758080c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:55.471868 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720986fa-2432-4780-a9f7-0a3e758080c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:55.473469 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720986fa-2432-4780-a9f7-0a3e758080c6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:55.485113 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1326/2621] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:21:55.489526 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5af476-ffe7-43a0-ab84-9e0f3e369c29 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:55.499956 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5af476-ffe7-43a0-ab84-9e0f3e369c29 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:55.504513 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf308e-82e5-4095-9bc5-2f6246aa0a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:55.505394 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf308e-82e5-4095-9bc5-2f6246aa0a1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:55.505674 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf308e-82e5-4095-9bc5-2f6246aa0a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:55.506736 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf308e-82e5-4095-9bc5-2f6246aa0a1b admin admin] PATH_INFO: `/v3/projects/336ae9e3da4142c794b33bd29a738c6b/users/4119fd46334647cfa262febd43acb3bd/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:55.508702 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bf308e-82e5-4095-9bc5-2f6246aa0a1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=336ae9e3da4142c794b33bd29a738c6b, user_id=4119fd46334647cfa262febd43acb3bd, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:55.511431 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bf308e-82e5-4095-9bc5-2f6246aa0a1b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:55.521434 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1296/2622] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:21:55 2022] PUT /identity/v3/projects/336ae9e3da4142c794b33bd29a738c6b/users/4119fd46334647cfa262febd43acb3bd/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:21:55.532441 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c904f7b-504e-4daa-9aa4-3b735fce42ec None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:55.532735 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c904f7b-504e-4daa-9aa4-3b735fce42ec None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:55.533041 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c904f7b-504e-4daa-9aa4-3b735fce42ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:55.560039 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7c904f7b-504e-4daa-9aa4-3b735fce42ec None None] MFA Rules not processed for user `4119fd46334647cfa262febd43acb3bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:21:55.561173 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7c904f7b-504e-4daa-9aa4-3b735fce42ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:55.593525 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:21:55.635987 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1327/2623] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:21:55 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:21:55.648213 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf308e-82e5-4095-9bc5-2f6246aa0a1b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:55.652103 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e5bf308e-82e5-4095-9bc5-2f6246aa0a1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:21:55.703791 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:21:55.751618 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf308e-82e5-4095-9bc5-2f6246aa0a1b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:55.756430 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e4cb54-0264-48c2-9a39-5402ee9dd6ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:55.757250 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4cb54-0264-48c2-9a39-5402ee9dd6ae service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:55.757674 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4cb54-0264-48c2-9a39-5402ee9dd6ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:55.758054 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4cb54-0264-48c2-9a39-5402ee9dd6ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:55.769629 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e4cb54-0264-48c2-9a39-5402ee9dd6ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:55.771270 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e4cb54-0264-48c2-9a39-5402ee9dd6ae service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:55.783079 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1297/2624] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:21:55 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:21:55.871326 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c904f7b-504e-4daa-9aa4-3b735fce42ec None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:21:55.884100 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c904f7b-504e-4daa-9aa4-3b735fce42ec None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:21:55.904840 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce058c84-893d-4dac-9c2c-dfb9ff4c8c77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:21:55.905670 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce058c84-893d-4dac-9c2c-dfb9ff4c8c77 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:21:55.906074 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce058c84-893d-4dac-9c2c-dfb9ff4c8c77 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:21:55.906453 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce058c84-893d-4dac-9c2c-dfb9ff4c8c77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:21:55.987018 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce058c84-893d-4dac-9c2c-dfb9ff4c8c77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:21:55.988877 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce058c84-893d-4dac-9c2c-dfb9ff4c8c77 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:21:56.004650 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1328/2625] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:21:55 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:22:01.874039 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6139b2c0-29df-4948-b89f-faafd2b773fe None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:01.874533 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6139b2c0-29df-4948-b89f-faafd2b773fe None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:01.874654 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6139b2c0-29df-4948-b89f-faafd2b773fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:01.894236 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-6139b2c0-29df-4948-b89f-faafd2b773fe None None] MFA Rules not processed for user `09c23d1c732c452a9597c647ad0241f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:22:01.895647 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6139b2c0-29df-4948-b89f-faafd2b773fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:01.957099 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1298/2626] 10.222.0.50 () {58 vars in 967 bytes} [Fri May 27 15:22:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:22:01.971017 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce058c84-893d-4dac-9c2c-dfb9ff4c8c77 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:01.972546 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ce058c84-893d-4dac-9c2c-dfb9ff4c8c77 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:02.048866 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce058c84-893d-4dac-9c2c-dfb9ff4c8c77 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:02.054558 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4611c255-36a9-464d-be6b-85a0f682c979 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18903f2eab7c415096e075d47f0b9a4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:02.055246 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4611c255-36a9-464d-be6b-85a0f682c979 service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:02.055531 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4611c255-36a9-464d-be6b-85a0f682c979 service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:02.055813 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4611c255-36a9-464d-be6b-85a0f682c979 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:02.065154 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4611c255-36a9-464d-be6b-85a0f682c979 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:02.066754 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4611c255-36a9-464d-be6b-85a0f682c979 service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:02.075979 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1329/2627] 10.222.0.50 () {60 vars in 1384 bytes} [Fri May 27 15:22:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 15:22:09.254251 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbb155-75aa-4562-835f-3b3f7c550f72 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:09.254721 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbb155-75aa-4562-835f-3b3f7c550f72 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:09.254820 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbb155-75aa-4562-835f-3b3f7c550f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:09.273400 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-38cbb155-75aa-4562-835f-3b3f7c550f72 None None] MFA Rules not processed for user `09c23d1c732c452a9597c647ad0241f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:22:09.274596 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-38cbb155-75aa-4562-835f-3b3f7c550f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:09.281638 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1299/2628] 10.222.0.50 () {58 vars in 1009 bytes} [Fri May 27 15:22:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:22:09.294714 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4611c255-36a9-464d-be6b-85a0f682c979 service swift] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:09.305451 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4611c255-36a9-464d-be6b-85a0f682c979 service swift] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:09.310492 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977548c3-40ec-48f9-b586-cc4d6e7029d6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18903f2eab7c415096e075d47f0b9a4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:09.311161 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977548c3-40ec-48f9-b586-cc4d6e7029d6 service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:09.311450 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977548c3-40ec-48f9-b586-cc4d6e7029d6 service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:09.311729 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977548c3-40ec-48f9-b586-cc4d6e7029d6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:09.319716 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977548c3-40ec-48f9-b586-cc4d6e7029d6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:09.321276 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977548c3-40ec-48f9-b586-cc4d6e7029d6 service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:09.329374 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1330/2629] 10.222.0.50 () {60 vars in 1384 bytes} [Fri May 27 15:22:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 27 15:22:09.872981 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077dbd5-5145-4615-9cbe-2b5521ad84e0 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:09.873267 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077dbd5-5145-4615-9cbe-2b5521ad84e0 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:09.873548 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077dbd5-5145-4615-9cbe-2b5521ad84e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:09.891619 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-1077dbd5-5145-4615-9cbe-2b5521ad84e0 None None] MFA Rules not processed for user `09c23d1c732c452a9597c647ad0241f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:22:09.892725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1077dbd5-5145-4615-9cbe-2b5521ad84e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:09.900306 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1300/2630] 10.222.0.50 () {58 vars in 967 bytes} [Fri May 27 15:22:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:22:09.912999 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977548c3-40ec-48f9-b586-cc4d6e7029d6 service swift] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:09.925379 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977548c3-40ec-48f9-b586-cc4d6e7029d6 service swift] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:09.931876 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8505dea4-ec3d-4092-b909-74a91fa6b6da service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18903f2eab7c415096e075d47f0b9a4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:09.932627 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8505dea4-ec3d-4092-b909-74a91fa6b6da service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:09.932946 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8505dea4-ec3d-4092-b909-74a91fa6b6da service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:09.933251 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8505dea4-ec3d-4092-b909-74a91fa6b6da service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:09.942876 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8505dea4-ec3d-4092-b909-74a91fa6b6da service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:09.945036 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8505dea4-ec3d-4092-b909-74a91fa6b6da service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:09.967747 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1331/2631] 10.222.0.50 () {60 vars in 1384 bytes} [Fri May 27 15:22:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 15:22:10.169624 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae16099-e763-4fc6-bb09-4849d0c57e07 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:10.169920 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae16099-e763-4fc6-bb09-4849d0c57e07 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:10.170203 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae16099-e763-4fc6-bb09-4849d0c57e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:10.189403 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3ae16099-e763-4fc6-bb09-4849d0c57e07 None None] MFA Rules not processed for user `09c23d1c732c452a9597c647ad0241f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:22:10.190606 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3ae16099-e763-4fc6-bb09-4849d0c57e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:10.198533 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1301/2632] 10.222.0.50 () {58 vars in 1009 bytes} [Fri May 27 15:22:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:22:10.211858 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8505dea4-ec3d-4092-b909-74a91fa6b6da service swift] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:10.224301 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8505dea4-ec3d-4092-b909-74a91fa6b6da service swift] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:10.231494 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f3f73-732c-4b2e-b8cc-4fbca9630528 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18903f2eab7c415096e075d47f0b9a4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:10.232236 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f3f73-732c-4b2e-b8cc-4fbca9630528 service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:10.232584 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f3f73-732c-4b2e-b8cc-4fbca9630528 service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:10.232938 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f3f73-732c-4b2e-b8cc-4fbca9630528 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:10.244283 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574f3f73-732c-4b2e-b8cc-4fbca9630528 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:10.246137 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574f3f73-732c-4b2e-b8cc-4fbca9630528 service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:10.256008 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1332/2633] 10.222.0.50 () {60 vars in 1384 bytes} [Fri May 27 15:22:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 15:22:10.592790 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cdad92-e324-4762-9975-82dd1b7931cb None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:10.593263 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cdad92-e324-4762-9975-82dd1b7931cb None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:10.593429 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cdad92-e324-4762-9975-82dd1b7931cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:10.614341 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-e9cdad92-e324-4762-9975-82dd1b7931cb None None] MFA Rules not processed for user `09c23d1c732c452a9597c647ad0241f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:22:10.615905 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e9cdad92-e324-4762-9975-82dd1b7931cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:10.627168 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1302/2634] 10.222.0.50 () {58 vars in 967 bytes} [Fri May 27 15:22:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:22:10.639768 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f3f73-732c-4b2e-b8cc-4fbca9630528 service swift] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:10.650743 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f3f73-732c-4b2e-b8cc-4fbca9630528 service swift] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:10.655707 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c9adab-22ac-4e87-a9b3-0a1e41cdeb35 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '18903f2eab7c415096e075d47f0b9a4d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:10.656338 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c9adab-22ac-4e87-a9b3-0a1e41cdeb35 service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:10.656617 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c9adab-22ac-4e87-a9b3-0a1e41cdeb35 service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:10.656892 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c9adab-22ac-4e87-a9b3-0a1e41cdeb35 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:10.665098 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c9adab-22ac-4e87-a9b3-0a1e41cdeb35 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:10.666604 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c9adab-22ac-4e87-a9b3-0a1e41cdeb35 service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:10.671114 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:22:10.675121 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1333/2635] 10.222.0.50 () {60 vars in 1384 bytes} [Fri May 27 15:22:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 27 15:22:11.146577 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cdad92-e324-4762-9975-82dd1b7931cb None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:11.223756 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cdad92-e324-4762-9975-82dd1b7931cb None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:11.229923 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53112e-80f7-4828-a099-db846c214b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:11.231340 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53112e-80f7-4828-a099-db846c214b04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:11.231774 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53112e-80f7-4828-a099-db846c214b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:11.232169 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa53112e-80f7-4828-a099-db846c214b04 admin admin] PATH_INFO: `/v3/users/ac93fff21b8a41faaad7008d354a3296` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:11.233916 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa53112e-80f7-4828-a099-db846c214b04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac93fff21b8a41faaad7008d354a3296)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:11.236957 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa53112e-80f7-4828-a099-db846c214b04 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:11.301060 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fa53112e-80f7-4828-a099-db846c214b04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac93fff21b8a41faaad7008d354a3296'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:11.306407 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:22:11.313657 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fa53112e-80f7-4828-a099-db846c214b04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac93fff21b8a41faaad7008d354a3296'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:11.326958 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fa53112e-80f7-4828-a099-db846c214b04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac93fff21b8a41faaad7008d354a3296'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:11.327948 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fa53112e-80f7-4828-a099-db846c214b04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac93fff21b8a41faaad7008d354a3296'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:11.328853 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fa53112e-80f7-4828-a099-db846c214b04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac93fff21b8a41faaad7008d354a3296'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:11.349099 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1303/2636] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:22:11 2022] DELETE /identity/v3/users/ac93fff21b8a41faaad7008d354a3296 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:22:11.354504 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f2f44-8569-4e27-b2a8-8e3e831948b1 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:11.354915 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f2f44-8569-4e27-b2a8-8e3e831948b1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:11.355300 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f2f44-8569-4e27-b2a8-8e3e831948b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:11.378514 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3c8f2f44-8569-4e27-b2a8-8e3e831948b1 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:22:11.380044 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3c8f2f44-8569-4e27-b2a8-8e3e831948b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:11.448704 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1334/2637] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:22:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:22:11.458268 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53112e-80f7-4828-a099-db846c214b04 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:11.459995 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fa53112e-80f7-4828-a099-db846c214b04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:11.523856 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa53112e-80f7-4828-a099-db846c214b04 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:11.527609 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e880fc2e-0360-413f-948e-78c2bf8bb366 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:11.528255 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e880fc2e-0360-413f-948e-78c2bf8bb366 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:11.528536 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e880fc2e-0360-413f-948e-78c2bf8bb366 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:11.528815 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e880fc2e-0360-413f-948e-78c2bf8bb366 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:11.540072 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e880fc2e-0360-413f-948e-78c2bf8bb366 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:11.541519 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e880fc2e-0360-413f-948e-78c2bf8bb366 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:11.552647 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1304/2638] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:22:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:22:11.858690 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f2f44-8569-4e27-b2a8-8e3e831948b1 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:11.860431 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3c8f2f44-8569-4e27-b2a8-8e3e831948b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:11.877560 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f2f44-8569-4e27-b2a8-8e3e831948b1 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:11.883319 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c629a-acee-4562-9cd8-fb94c33418e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:11.884406 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c629a-acee-4562-9cd8-fb94c33418e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:11.884857 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c629a-acee-4562-9cd8-fb94c33418e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:11.885016 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c629a-acee-4562-9cd8-fb94c33418e7 admin admin] PATH_INFO: `/v3/projects/76222af4f3af4f24bfec86bc29ddd65c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:11.886336 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038c629a-acee-4562-9cd8-fb94c33418e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76222af4f3af4f24bfec86bc29ddd65c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:11.888164 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038c629a-acee-4562-9cd8-fb94c33418e7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:11.938214 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-038c629a-acee-4562-9cd8-fb94c33418e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76222af4f3af4f24bfec86bc29ddd65c', 'request_id': 'req-038c629a-acee-4562-9cd8-fb94c33418e7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:11.946394 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-038c629a-acee-4562-9cd8-fb94c33418e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76222af4f3af4f24bfec86bc29ddd65c', 'request_id': 'req-038c629a-acee-4562-9cd8-fb94c33418e7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:11.952371 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-038c629a-acee-4562-9cd8-fb94c33418e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76222af4f3af4f24bfec86bc29ddd65c', 'request_id': 'req-038c629a-acee-4562-9cd8-fb94c33418e7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:11.966231 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-038c629a-acee-4562-9cd8-fb94c33418e7 admin admin] The token cache is being invalidate because project 76222af4f3af4f24bfec86bc29ddd65c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:22:11.966544 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-038c629a-acee-4562-9cd8-fb94c33418e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:11.967599 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-038c629a-acee-4562-9cd8-fb94c33418e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:11.970345 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1335/2639] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:22:11 2022] DELETE /identity/v3/projects/76222af4f3af4f24bfec86bc29ddd65c => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:22:11.975864 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e880fc2e-0360-413f-948e-78c2bf8bb366 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:11.986481 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e880fc2e-0360-413f-948e-78c2bf8bb366 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:12.048346 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6f1b26-9920-4a64-9078-fe047a752524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:12.049223 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f1b26-9920-4a64-9078-fe047a752524 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:12.049502 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f1b26-9920-4a64-9078-fe047a752524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:12.049776 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f1b26-9920-4a64-9078-fe047a752524 admin admin] PATH_INFO: `/v3/users/4119fd46334647cfa262febd43acb3bd` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:12.051051 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6f1b26-9920-4a64-9078-fe047a752524 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4119fd46334647cfa262febd43acb3bd)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:12.053050 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6f1b26-9920-4a64-9078-fe047a752524 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:12.103419 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5c6f1b26-9920-4a64-9078-fe047a752524 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4119fd46334647cfa262febd43acb3bd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:12.111056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5c6f1b26-9920-4a64-9078-fe047a752524 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4119fd46334647cfa262febd43acb3bd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:12.119134 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5c6f1b26-9920-4a64-9078-fe047a752524 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4119fd46334647cfa262febd43acb3bd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:12.119775 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5c6f1b26-9920-4a64-9078-fe047a752524 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4119fd46334647cfa262febd43acb3bd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:12.120416 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-5c6f1b26-9920-4a64-9078-fe047a752524 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4119fd46334647cfa262febd43acb3bd'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:12.128232 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:22:12.140864 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1305/2640] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:22:11 2022] DELETE /identity/v3/users/4119fd46334647cfa262febd43acb3bd => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:22:12.418945 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c629a-acee-4562-9cd8-fb94c33418e7 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:12.420442 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-038c629a-acee-4562-9cd8-fb94c33418e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:12.435283 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c629a-acee-4562-9cd8-fb94c33418e7 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:12.492810 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de386c3-994d-4bd0-a72d-3c9004e5e54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:12.493680 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de386c3-994d-4bd0-a72d-3c9004e5e54f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:12.493966 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de386c3-994d-4bd0-a72d-3c9004e5e54f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:12.494246 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de386c3-994d-4bd0-a72d-3c9004e5e54f admin admin] PATH_INFO: `/v3/projects/336ae9e3da4142c794b33bd29a738c6b` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:12.495517 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de386c3-994d-4bd0-a72d-3c9004e5e54f admin admin] RBAC: Authorizing `identity:delete_project(project_id=336ae9e3da4142c794b33bd29a738c6b)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:12.497655 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de386c3-994d-4bd0-a72d-3c9004e5e54f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:12.528820 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:22:12.547047 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9de386c3-994d-4bd0-a72d-3c9004e5e54f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '336ae9e3da4142c794b33bd29a738c6b', 'request_id': 'req-9de386c3-994d-4bd0-a72d-3c9004e5e54f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:12.553777 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9de386c3-994d-4bd0-a72d-3c9004e5e54f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '336ae9e3da4142c794b33bd29a738c6b', 'request_id': 'req-9de386c3-994d-4bd0-a72d-3c9004e5e54f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:12.559588 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9de386c3-994d-4bd0-a72d-3c9004e5e54f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '336ae9e3da4142c794b33bd29a738c6b', 'request_id': 'req-9de386c3-994d-4bd0-a72d-3c9004e5e54f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:12.571261 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9de386c3-994d-4bd0-a72d-3c9004e5e54f admin admin] The token cache is being invalidate because project 336ae9e3da4142c794b33bd29a738c6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:22:12.571545 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9de386c3-994d-4bd0-a72d-3c9004e5e54f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:12.572272 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9de386c3-994d-4bd0-a72d-3c9004e5e54f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:12.573861 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1336/2641] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:22:12 2022] DELETE /identity/v3/projects/336ae9e3da4142c794b33bd29a738c6b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:22:12.578671 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5998fe0-ffcb-4fa3-a396-b13945874f1c None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:12.578979 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5998fe0-ffcb-4fa3-a396-b13945874f1c None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:12.579255 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5998fe0-ffcb-4fa3-a396-b13945874f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:12.598667 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f5998fe0-ffcb-4fa3-a396-b13945874f1c None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:22:12.599883 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f5998fe0-ffcb-4fa3-a396-b13945874f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:12.661572 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1306/2642] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:22:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:22:12.670337 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148748df-d6d9-486f-8655-2111e6470345 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:12.670629 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148748df-d6d9-486f-8655-2111e6470345 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:12.670950 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148748df-d6d9-486f-8655-2111e6470345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:12.689432 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-148748df-d6d9-486f-8655-2111e6470345 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:22:12.690626 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-148748df-d6d9-486f-8655-2111e6470345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:12.699120 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1337/2643] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:22:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:22:12.703333 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5998fe0-ffcb-4fa3-a396-b13945874f1c None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:12.714029 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5998fe0-ffcb-4fa3-a396-b13945874f1c None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:12.718620 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e249aa-3683-4446-9760-40146923d27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:12.719305 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e249aa-3683-4446-9760-40146923d27a admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:12.719858 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e249aa-3683-4446-9760-40146923d27a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:12.720136 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e249aa-3683-4446-9760-40146923d27a admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:12.720884 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e249aa-3683-4446-9760-40146923d27a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:22:12.721193 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e249aa-3683-4446-9760-40146923d27a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:12.722787 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e249aa-3683-4446-9760-40146923d27a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:12.734705 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1307/2644] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:22:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:22:12.739111 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148748df-d6d9-486f-8655-2111e6470345 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:12.748946 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148748df-d6d9-486f-8655-2111e6470345 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:12.753938 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148a1ed5-0de0-47b5-a19c-2b4276b0b070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:12.754608 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148a1ed5-0de0-47b5-a19c-2b4276b0b070 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:12.754926 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148a1ed5-0de0-47b5-a19c-2b4276b0b070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:12.755206 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148a1ed5-0de0-47b5-a19c-2b4276b0b070 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:12.755908 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148a1ed5-0de0-47b5-a19c-2b4276b0b070 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:12.758646 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148a1ed5-0de0-47b5-a19c-2b4276b0b070 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:12.786476 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1338/2645] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:22:12 2022] POST /identity/v3/projects => generated 363 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:22:12.790930 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e249aa-3683-4446-9760-40146923d27a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:12.800838 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e249aa-3683-4446-9760-40146923d27a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:12.831861 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148a1ed5-0de0-47b5-a19c-2b4276b0b070 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:12.834686 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-148a1ed5-0de0-47b5-a19c-2b4276b0b070 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:12.861015 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5872df6-d6b6-464c-b447-04358378f6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:12.861685 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5872df6-d6b6-464c-b447-04358378f6d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:12.861971 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5872df6-d6b6-464c-b447-04358378f6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:12.862249 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5872df6-d6b6-464c-b447-04358378f6d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:12.862965 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5872df6-d6b6-464c-b447-04358378f6d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:12.864909 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5872df6-d6b6-464c-b447-04358378f6d5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:12.885891 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1308/2646] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:22:12 2022] POST /identity/v3/users => generated 282 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:22:12.889922 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5872df6-d6b6-464c-b447-04358378f6d5 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:12.900576 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5872df6-d6b6-464c-b447-04358378f6d5 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:12.905252 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabafe71-3a65-483c-95c8-a7be4d58f24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:12.905858 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabafe71-3a65-483c-95c8-a7be4d58f24a admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:12.906139 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabafe71-3a65-483c-95c8-a7be4d58f24a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:12.906413 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabafe71-3a65-483c-95c8-a7be4d58f24a admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:12.907190 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabafe71-3a65-483c-95c8-a7be4d58f24a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:12.908768 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabafe71-3a65-483c-95c8-a7be4d58f24a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:12.915661 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148a1ed5-0de0-47b5-a19c-2b4276b0b070 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:12.919766 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1309/2647] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:22:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:22:12.920566 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceb342d-aec7-40ad-8616-b56aea8c35e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:12.921443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb342d-aec7-40ad-8616-b56aea8c35e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:12.921822 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb342d-aec7-40ad-8616-b56aea8c35e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:12.922103 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb342d-aec7-40ad-8616-b56aea8c35e1 admin admin] PATH_INFO: `/v3/users/4cb25251cbd442a89f0a73bf35e76c3c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:12.923379 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceb342d-aec7-40ad-8616-b56aea8c35e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cb25251cbd442a89f0a73bf35e76c3c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:12.923677 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabafe71-3a65-483c-95c8-a7be4d58f24a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:12.925305 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceb342d-aec7-40ad-8616-b56aea8c35e1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:12.934050 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabafe71-3a65-483c-95c8-a7be4d58f24a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:12.938823 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d945c9c4-2e59-4a0b-a800-b505088a91c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:12.939511 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d945c9c4-2e59-4a0b-a800-b505088a91c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:12.939789 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d945c9c4-2e59-4a0b-a800-b505088a91c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:12.940067 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d945c9c4-2e59-4a0b-a800-b505088a91c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:12.940778 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d945c9c4-2e59-4a0b-a800-b505088a91c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:12.942363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d945c9c4-2e59-4a0b-a800-b505088a91c8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:12.954573 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1310/2648] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:22:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:22:12.959071 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d945c9c4-2e59-4a0b-a800-b505088a91c8 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:12.970025 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d945c9c4-2e59-4a0b-a800-b505088a91c8 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:12.974805 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f50932-ad99-4dd0-a437-91b69030c3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:12.976233 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f50932-ad99-4dd0-a437-91b69030c3be admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:12.976902 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f50932-ad99-4dd0-a437-91b69030c3be admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:12.976902 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f50932-ad99-4dd0-a437-91b69030c3be admin admin] PATH_INFO: `/v3/projects/f0ea6b42c787449cabdfa763bd443ffa/users/4ebb256a9a754b3bae20f72fafe905c5/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:12.979204 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6ceb342d-aec7-40ad-8616-b56aea8c35e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cb25251cbd442a89f0a73bf35e76c3c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:12.987581 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6ceb342d-aec7-40ad-8616-b56aea8c35e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cb25251cbd442a89f0a73bf35e76c3c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:12.988483 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6ceb342d-aec7-40ad-8616-b56aea8c35e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cb25251cbd442a89f0a73bf35e76c3c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:12.993942 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f50932-ad99-4dd0-a437-91b69030c3be admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0ea6b42c787449cabdfa763bd443ffa, user_id=4ebb256a9a754b3bae20f72fafe905c5, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:12.996929 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f50932-ad99-4dd0-a437-91b69030c3be admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:12.997353 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6ceb342d-aec7-40ad-8616-b56aea8c35e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cb25251cbd442a89f0a73bf35e76c3c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:12.998115 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6ceb342d-aec7-40ad-8616-b56aea8c35e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cb25251cbd442a89f0a73bf35e76c3c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:13.010635 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1311/2649] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:22:12 2022] PUT /identity/v3/projects/f0ea6b42c787449cabdfa763bd443ffa/users/4ebb256a9a754b3bae20f72fafe905c5/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:22:13.020941 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1339/2650] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:22:12 2022] DELETE /identity/v3/users/4cb25251cbd442a89f0a73bf35e76c3c => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:22:13.022237 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec840045-1326-480f-bf63-57d0bf713a2b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:13.023412 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec840045-1326-480f-bf63-57d0bf713a2b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:13.023822 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec840045-1326-480f-bf63-57d0bf713a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:13.025866 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e59c9b-5702-4269-89ad-433abba33547 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:13.026206 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e59c9b-5702-4269-89ad-433abba33547 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:13.026524 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e59c9b-5702-4269-89ad-433abba33547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:13.033140 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:DELETE=16,keystone:SELECT=81,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:22:13.047540 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-13e59c9b-5702-4269-89ad-433abba33547 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:22:13.048796 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-13e59c9b-5702-4269-89ad-433abba33547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:13.056727 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-ec840045-1326-480f-bf63-57d0bf713a2b None None] MFA Rules not processed for user `4ebb256a9a754b3bae20f72fafe905c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:22:13.057923 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ec840045-1326-480f-bf63-57d0bf713a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:13.067703 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:DELETE=1,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:22:13.110407 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1340/2651] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:22:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:22:13.121926 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e59c9b-5702-4269-89ad-433abba33547 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:13.123622 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-13e59c9b-5702-4269-89ad-433abba33547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:13.127144 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1312/2652] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:22:13 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:22:13.131199 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec840045-1326-480f-bf63-57d0bf713a2b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:13.133619 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-ec840045-1326-480f-bf63-57d0bf713a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:13.148598 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec840045-1326-480f-bf63-57d0bf713a2b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:13.153442 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0572290b-489e-455e-a94e-db95c08e18c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:13.154104 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0572290b-489e-455e-a94e-db95c08e18c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:13.154385 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0572290b-489e-455e-a94e-db95c08e18c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:13.154659 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0572290b-489e-455e-a94e-db95c08e18c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:13.155508 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0572290b-489e-455e-a94e-db95c08e18c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:13.160469 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0572290b-489e-455e-a94e-db95c08e18c1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:13.188170 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1313/2653] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:22:13 2022] POST /identity/v3/projects => generated 361 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:22:13.189133 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0572290b-489e-455e-a94e-db95c08e18c1 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:13.194839 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e59c9b-5702-4269-89ad-433abba33547 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:13.252833 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f23024-025d-4dc9-9396-2cbaa87dbd6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:13.253755 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f23024-025d-4dc9-9396-2cbaa87dbd6f service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:13.254232 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f23024-025d-4dc9-9396-2cbaa87dbd6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:13.254643 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0572290b-489e-455e-a94e-db95c08e18c1 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:13.254795 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f23024-025d-4dc9-9396-2cbaa87dbd6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:13.260037 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b83ca6-8d5b-4042-b3d3-3285b762eff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:13.260765 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b83ca6-8d5b-4042-b3d3-3285b762eff1 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:13.260999 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b83ca6-8d5b-4042-b3d3-3285b762eff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:13.261276 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b83ca6-8d5b-4042-b3d3-3285b762eff1 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:13.261963 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b83ca6-8d5b-4042-b3d3-3285b762eff1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:13.263939 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b83ca6-8d5b-4042-b3d3-3285b762eff1 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:13.266712 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f23024-025d-4dc9-9396-2cbaa87dbd6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:13.268651 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f23024-025d-4dc9-9396-2cbaa87dbd6f service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:13.283468 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1341/2654] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:22:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:22:13.290160 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1314/2655] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:22:13 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:22:13.294650 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f23024-025d-4dc9-9396-2cbaa87dbd6f service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:13.308435 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f23024-025d-4dc9-9396-2cbaa87dbd6f service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:13.313348 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff50e0a-c208-4b1e-b364-20464ef6929a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:13.314036 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff50e0a-c208-4b1e-b364-20464ef6929a admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:13.314704 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff50e0a-c208-4b1e-b364-20464ef6929a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:13.314704 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff50e0a-c208-4b1e-b364-20464ef6929a admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:13.315443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff50e0a-c208-4b1e-b364-20464ef6929a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:13.317082 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff50e0a-c208-4b1e-b364-20464ef6929a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:13.328177 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1342/2656] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:22:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:22:13.332636 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b83ca6-8d5b-4042-b3d3-3285b762eff1 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:13.344789 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b83ca6-8d5b-4042-b3d3-3285b762eff1 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:13.350205 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ddbda7-5bcf-48c9-9019-fe2b6b98a568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:13.351403 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ddbda7-5bcf-48c9-9019-fe2b6b98a568 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:13.351779 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ddbda7-5bcf-48c9-9019-fe2b6b98a568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:13.352118 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ddbda7-5bcf-48c9-9019-fe2b6b98a568 admin admin] PATH_INFO: `/v3/projects/4dc09e0c8b97401096002f924b7c7c2e/users/cd2260f0494b43b39cf7b836c78db828/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:13.370083 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ddbda7-5bcf-48c9-9019-fe2b6b98a568 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dc09e0c8b97401096002f924b7c7c2e, user_id=cd2260f0494b43b39cf7b836c78db828, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:13.373149 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ddbda7-5bcf-48c9-9019-fe2b6b98a568 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:13.388259 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1315/2657] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:22:13 2022] PUT /identity/v3/projects/4dc09e0c8b97401096002f924b7c7c2e/users/cd2260f0494b43b39cf7b836c78db828/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:22:13.392653 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff50e0a-c208-4b1e-b364-20464ef6929a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:13.402766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff50e0a-c208-4b1e-b364-20464ef6929a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:13.452326 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:22:13.462247 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e805d295-da8a-4409-bfa0-bed492d037b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:13.462994 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e805d295-da8a-4409-bfa0-bed492d037b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:13.463468 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e805d295-da8a-4409-bfa0-bed492d037b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:13.463776 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e805d295-da8a-4409-bfa0-bed492d037b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:13.464536 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e805d295-da8a-4409-bfa0-bed492d037b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:13.466621 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e805d295-da8a-4409-bfa0-bed492d037b3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:13.482881 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1343/2658] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:22:13 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:22:13.484657 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ddbda7-5bcf-48c9-9019-fe2b6b98a568 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:13.496999 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e805d295-da8a-4409-bfa0-bed492d037b3 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:13.499182 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e805d295-da8a-4409-bfa0-bed492d037b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:13.499900 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ddbda7-5bcf-48c9-9019-fe2b6b98a568 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:13.504744 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4efd7bd-5652-46be-9bb1-3b0ca734e939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:13.505688 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4efd7bd-5652-46be-9bb1-3b0ca734e939 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:13.505989 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4efd7bd-5652-46be-9bb1-3b0ca734e939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:13.506286 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4efd7bd-5652-46be-9bb1-3b0ca734e939 admin admin] PATH_INFO: `/v3/projects/4dc09e0c8b97401096002f924b7c7c2e/users/cd2260f0494b43b39cf7b836c78db828/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:13.510404 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4efd7bd-5652-46be-9bb1-3b0ca734e939 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dc09e0c8b97401096002f924b7c7c2e, user_id=cd2260f0494b43b39cf7b836c78db828, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:13.512804 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e805d295-da8a-4409-bfa0-bed492d037b3 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:13.513001 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4efd7bd-5652-46be-9bb1-3b0ca734e939 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:13.517325 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f67d00c-b01f-45a6-8ddc-99f379c89fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:13.518224 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67d00c-b01f-45a6-8ddc-99f379c89fa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:13.518980 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67d00c-b01f-45a6-8ddc-99f379c89fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:13.518980 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f67d00c-b01f-45a6-8ddc-99f379c89fa2 admin admin] PATH_INFO: `/v3/projects/9c3ebf345dbe46f7a137e1a62dee5f33` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:13.520434 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f67d00c-b01f-45a6-8ddc-99f379c89fa2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c3ebf345dbe46f7a137e1a62dee5f33)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:13.522544 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f67d00c-b01f-45a6-8ddc-99f379c89fa2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:13.527231 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1316/2659] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:22:13 2022] PUT /identity/v3/projects/4dc09e0c8b97401096002f924b7c7c2e/users/cd2260f0494b43b39cf7b836c78db828/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:22:13.540540 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bdfa7-71f3-4594-bee8-4f51f7889124 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:13.540946 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bdfa7-71f3-4594-bee8-4f51f7889124 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:13.541236 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bdfa7-71f3-4594-bee8-4f51f7889124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:13.569522 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-c58bdfa7-71f3-4594-bee8-4f51f7889124 None None] MFA Rules not processed for user `cd2260f0494b43b39cf7b836c78db828`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:22:13.570835 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c58bdfa7-71f3-4594-bee8-4f51f7889124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:13.572955 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2f67d00c-b01f-45a6-8ddc-99f379c89fa2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c3ebf345dbe46f7a137e1a62dee5f33', 'request_id': 'req-2f67d00c-b01f-45a6-8ddc-99f379c89fa2'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:13.580923 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2f67d00c-b01f-45a6-8ddc-99f379c89fa2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c3ebf345dbe46f7a137e1a62dee5f33', 'request_id': 'req-2f67d00c-b01f-45a6-8ddc-99f379c89fa2'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:13.587336 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2f67d00c-b01f-45a6-8ddc-99f379c89fa2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c3ebf345dbe46f7a137e1a62dee5f33', 'request_id': 'req-2f67d00c-b01f-45a6-8ddc-99f379c89fa2'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:13.600383 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2f67d00c-b01f-45a6-8ddc-99f379c89fa2 admin admin] The token cache is being invalidate because project 9c3ebf345dbe46f7a137e1a62dee5f33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:22:13.601379 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2f67d00c-b01f-45a6-8ddc-99f379c89fa2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:13.602455 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2f67d00c-b01f-45a6-8ddc-99f379c89fa2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:13.604772 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1344/2660] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:22:13 2022] DELETE /identity/v3/projects/9c3ebf345dbe46f7a137e1a62dee5f33 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:22:13.610799 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:22:13.655924 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1317/2661] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:22:13 2022] POST /identity/v3/auth/tokens => generated 3295 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:22:13.669149 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f67d00c-b01f-45a6-8ddc-99f379c89fa2 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:13.670804 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2f67d00c-b01f-45a6-8ddc-99f379c89fa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:13.739430 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f67d00c-b01f-45a6-8ddc-99f379c89fa2 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:13.744008 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f40680-e0d8-4c5a-b10e-97ded5f86706 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:13.744707 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f40680-e0d8-4c5a-b10e-97ded5f86706 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:13.745015 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f40680-e0d8-4c5a-b10e-97ded5f86706 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:13.745322 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f40680-e0d8-4c5a-b10e-97ded5f86706 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:13.755877 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f40680-e0d8-4c5a-b10e-97ded5f86706 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:13.757630 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f40680-e0d8-4c5a-b10e-97ded5f86706 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:13.768891 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1345/2662] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:22:13 2022] GET /identity/v3/auth/tokens => generated 3295 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:22:13.857487 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58bdfa7-71f3-4594-bee8-4f51f7889124 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:13.868687 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58bdfa7-71f3-4594-bee8-4f51f7889124 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:13.872550 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0230421-116f-47bd-b623-f2763fc3635a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:13.873271 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0230421-116f-47bd-b623-f2763fc3635a service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:13.873579 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0230421-116f-47bd-b623-f2763fc3635a service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:13.873905 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0230421-116f-47bd-b623-f2763fc3635a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:13.875861 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c0230421-116f-47bd-b623-f2763fc3635a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:13.940365 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0230421-116f-47bd-b623-f2763fc3635a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:13.942098 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0230421-116f-47bd-b623-f2763fc3635a service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:13.953270 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1318/2663] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:22:13 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:22:16.683680 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f40680-e0d8-4c5a-b10e-97ded5f86706 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:16.685105 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-41f40680-e0d8-4c5a-b10e-97ded5f86706 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:16.752200 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f40680-e0d8-4c5a-b10e-97ded5f86706 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:16.756818 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb05b0a-be0a-4c2b-9a7f-420e748b1d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:16.757646 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb05b0a-be0a-4c2b-9a7f-420e748b1d51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:16.757929 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb05b0a-be0a-4c2b-9a7f-420e748b1d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:16.758206 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb05b0a-be0a-4c2b-9a7f-420e748b1d51 admin admin] PATH_INFO: `/v3/users/4ebb256a9a754b3bae20f72fafe905c5` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:16.759418 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb05b0a-be0a-4c2b-9a7f-420e748b1d51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ebb256a9a754b3bae20f72fafe905c5)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:16.761326 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb05b0a-be0a-4c2b-9a7f-420e748b1d51 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:16.777872 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:22:16.808654 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4cb05b0a-be0a-4c2b-9a7f-420e748b1d51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ebb256a9a754b3bae20f72fafe905c5'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:16.815393 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4cb05b0a-be0a-4c2b-9a7f-420e748b1d51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ebb256a9a754b3bae20f72fafe905c5'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:16.815991 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4cb05b0a-be0a-4c2b-9a7f-420e748b1d51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ebb256a9a754b3bae20f72fafe905c5'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:16.823496 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4cb05b0a-be0a-4c2b-9a7f-420e748b1d51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ebb256a9a754b3bae20f72fafe905c5'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:16.824103 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4cb05b0a-be0a-4c2b-9a7f-420e748b1d51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ebb256a9a754b3bae20f72fafe905c5'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:16.841788 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1346/2664] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:22:16 2022] DELETE /identity/v3/users/4ebb256a9a754b3bae20f72fafe905c5 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:22:16.847579 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d103dcc-ba46-4167-9118-34d48fb29719 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:16.848048 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d103dcc-ba46-4167-9118-34d48fb29719 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:16.848282 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d103dcc-ba46-4167-9118-34d48fb29719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:16.868121 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-7d103dcc-ba46-4167-9118-34d48fb29719 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:22:16.869309 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7d103dcc-ba46-4167-9118-34d48fb29719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:16.927534 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1319/2665] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:22:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:22:16.937226 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb05b0a-be0a-4c2b-9a7f-420e748b1d51 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:16.938631 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4cb05b0a-be0a-4c2b-9a7f-420e748b1d51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:16.997817 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb05b0a-be0a-4c2b-9a7f-420e748b1d51 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:17.001558 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3813bcde-40ec-4efa-b1b4-e02d0878d1a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:17.002208 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813bcde-40ec-4efa-b1b4-e02d0878d1a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:17.002493 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813bcde-40ec-4efa-b1b4-e02d0878d1a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:17.002767 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3813bcde-40ec-4efa-b1b4-e02d0878d1a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:17.012445 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3813bcde-40ec-4efa-b1b4-e02d0878d1a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:17.013894 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3813bcde-40ec-4efa-b1b4-e02d0878d1a9 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:17.025371 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1347/2666] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:22:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:22:17.301839 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d103dcc-ba46-4167-9118-34d48fb29719 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:17.303528 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7d103dcc-ba46-4167-9118-34d48fb29719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:17.316370 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d103dcc-ba46-4167-9118-34d48fb29719 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:17.321491 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6ce933-7ad7-46fd-bc6a-61f66aa65aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:17.322362 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ce933-7ad7-46fd-bc6a-61f66aa65aa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:17.322773 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ce933-7ad7-46fd-bc6a-61f66aa65aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:17.323084 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ce933-7ad7-46fd-bc6a-61f66aa65aa5 admin admin] PATH_INFO: `/v3/projects/f0ea6b42c787449cabdfa763bd443ffa` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:17.324278 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6ce933-7ad7-46fd-bc6a-61f66aa65aa5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0ea6b42c787449cabdfa763bd443ffa)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:17.326456 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6ce933-7ad7-46fd-bc6a-61f66aa65aa5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:17.379983 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3d6ce933-7ad7-46fd-bc6a-61f66aa65aa5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0ea6b42c787449cabdfa763bd443ffa', 'request_id': 'req-3d6ce933-7ad7-46fd-bc6a-61f66aa65aa5'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:17.385991 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3d6ce933-7ad7-46fd-bc6a-61f66aa65aa5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0ea6b42c787449cabdfa763bd443ffa', 'request_id': 'req-3d6ce933-7ad7-46fd-bc6a-61f66aa65aa5'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:17.387914 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:22:17.394055 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3d6ce933-7ad7-46fd-bc6a-61f66aa65aa5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0ea6b42c787449cabdfa763bd443ffa', 'request_id': 'req-3d6ce933-7ad7-46fd-bc6a-61f66aa65aa5'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:17.408334 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3d6ce933-7ad7-46fd-bc6a-61f66aa65aa5 admin admin] The token cache is being invalidate because project f0ea6b42c787449cabdfa763bd443ffa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:22:17.408648 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3d6ce933-7ad7-46fd-bc6a-61f66aa65aa5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:17.409275 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3d6ce933-7ad7-46fd-bc6a-61f66aa65aa5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:17.410826 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1320/2667] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:22:17 2022] DELETE /identity/v3/projects/f0ea6b42c787449cabdfa763bd443ffa => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:22:17.415376 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3813bcde-40ec-4efa-b1b4-e02d0878d1a9 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:17.424910 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3813bcde-40ec-4efa-b1b4-e02d0878d1a9 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:17.486614 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dbcbe8-27ef-48f1-89ad-7f9561f444ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:17.487610 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbcbe8-27ef-48f1-89ad-7f9561f444ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:17.487919 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbcbe8-27ef-48f1-89ad-7f9561f444ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:17.488231 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbcbe8-27ef-48f1-89ad-7f9561f444ce admin admin] PATH_INFO: `/v3/users/cd2260f0494b43b39cf7b836c78db828` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:17.489550 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dbcbe8-27ef-48f1-89ad-7f9561f444ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd2260f0494b43b39cf7b836c78db828)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:17.492797 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dbcbe8-27ef-48f1-89ad-7f9561f444ce admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:17.536725 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:22:17.544652 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d7dbcbe8-27ef-48f1-89ad-7f9561f444ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd2260f0494b43b39cf7b836c78db828'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:17.552166 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d7dbcbe8-27ef-48f1-89ad-7f9561f444ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd2260f0494b43b39cf7b836c78db828'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:17.552841 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d7dbcbe8-27ef-48f1-89ad-7f9561f444ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd2260f0494b43b39cf7b836c78db828'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:17.560364 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d7dbcbe8-27ef-48f1-89ad-7f9561f444ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd2260f0494b43b39cf7b836c78db828'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:17.560984 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d7dbcbe8-27ef-48f1-89ad-7f9561f444ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd2260f0494b43b39cf7b836c78db828'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:17.578274 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1348/2668] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:22:17 2022] DELETE /identity/v3/users/cd2260f0494b43b39cf7b836c78db828 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:22:17.855907 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6ce933-7ad7-46fd-bc6a-61f66aa65aa5 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:17.857331 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3d6ce933-7ad7-46fd-bc6a-61f66aa65aa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:17.870500 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6ce933-7ad7-46fd-bc6a-61f66aa65aa5 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:17.930356 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb30a61-81cc-4eb3-82d4-6e70baaa4635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:17.931171 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb30a61-81cc-4eb3-82d4-6e70baaa4635 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:17.931457 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb30a61-81cc-4eb3-82d4-6e70baaa4635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:17.931734 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb30a61-81cc-4eb3-82d4-6e70baaa4635 admin admin] PATH_INFO: `/v3/projects/4dc09e0c8b97401096002f924b7c7c2e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:17.932923 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb30a61-81cc-4eb3-82d4-6e70baaa4635 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4dc09e0c8b97401096002f924b7c7c2e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:17.934828 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb30a61-81cc-4eb3-82d4-6e70baaa4635 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:17.982425 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-dbb30a61-81cc-4eb3-82d4-6e70baaa4635 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4dc09e0c8b97401096002f924b7c7c2e', 'request_id': 'req-dbb30a61-81cc-4eb3-82d4-6e70baaa4635'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:17.989041 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-dbb30a61-81cc-4eb3-82d4-6e70baaa4635 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4dc09e0c8b97401096002f924b7c7c2e', 'request_id': 'req-dbb30a61-81cc-4eb3-82d4-6e70baaa4635'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:17.995392 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-dbb30a61-81cc-4eb3-82d4-6e70baaa4635 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4dc09e0c8b97401096002f924b7c7c2e', 'request_id': 'req-dbb30a61-81cc-4eb3-82d4-6e70baaa4635'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:18.009419 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-dbb30a61-81cc-4eb3-82d4-6e70baaa4635 admin admin] The token cache is being invalidate because project 4dc09e0c8b97401096002f924b7c7c2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:22:18.009738 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-dbb30a61-81cc-4eb3-82d4-6e70baaa4635 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:18.010361 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-dbb30a61-81cc-4eb3-82d4-6e70baaa4635 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:18.011954 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1321/2669] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:22:17 2022] DELETE /identity/v3/projects/4dc09e0c8b97401096002f924b7c7c2e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:22:18.016819 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd68f72-8f62-4e89-8e65-c65a20aa2f99 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:18.017281 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd68f72-8f62-4e89-8e65-c65a20aa2f99 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:18.017397 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd68f72-8f62-4e89-8e65-c65a20aa2f99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:18.036032 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-ecd68f72-8f62-4e89-8e65-c65a20aa2f99 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:22:18.037233 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ecd68f72-8f62-4e89-8e65-c65a20aa2f99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:18.097188 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1349/2670] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:22:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:22:18.106631 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b39a5c-fd39-4d92-9766-e648cf1aea51 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:18.107010 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b39a5c-fd39-4d92-9766-e648cf1aea51 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:18.107376 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b39a5c-fd39-4d92-9766-e648cf1aea51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:18.127306 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-62b39a5c-fd39-4d92-9766-e648cf1aea51 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:22:18.128669 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-62b39a5c-fd39-4d92-9766-e648cf1aea51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:18.138223 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1322/2671] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:22:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:22:18.142962 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd68f72-8f62-4e89-8e65-c65a20aa2f99 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:18.154140 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd68f72-8f62-4e89-8e65-c65a20aa2f99 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:18.158783 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e119b5b1-3795-463b-96a6-51dcce43504e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:18.159561 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119b5b1-3795-463b-96a6-51dcce43504e admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:18.159868 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119b5b1-3795-463b-96a6-51dcce43504e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:18.160137 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e119b5b1-3795-463b-96a6-51dcce43504e admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:18.160857 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e119b5b1-3795-463b-96a6-51dcce43504e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:22:18.161161 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e119b5b1-3795-463b-96a6-51dcce43504e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:18.162738 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e119b5b1-3795-463b-96a6-51dcce43504e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:18.175276 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1350/2672] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:22:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:22:18.180933 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b39a5c-fd39-4d92-9766-e648cf1aea51 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:18.191484 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b39a5c-fd39-4d92-9766-e648cf1aea51 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:18.196870 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9e1970-9c4a-4a24-af27-306f92ce80e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:18.197572 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9e1970-9c4a-4a24-af27-306f92ce80e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:18.197880 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9e1970-9c4a-4a24-af27-306f92ce80e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:18.198200 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9e1970-9c4a-4a24-af27-306f92ce80e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:18.199010 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9e1970-9c4a-4a24-af27-306f92ce80e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:18.201131 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9e1970-9c4a-4a24-af27-306f92ce80e2 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:18.228696 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1323/2673] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:22:18 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:22:18.233111 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e119b5b1-3795-463b-96a6-51dcce43504e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:18.244818 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e119b5b1-3795-463b-96a6-51dcce43504e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:18.311370 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48519d23-acef-4c42-b80e-e96b44b87747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:18.312109 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48519d23-acef-4c42-b80e-e96b44b87747 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:18.312423 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48519d23-acef-4c42-b80e-e96b44b87747 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:18.312731 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48519d23-acef-4c42-b80e-e96b44b87747 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:18.313500 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48519d23-acef-4c42-b80e-e96b44b87747 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:18.316301 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48519d23-acef-4c42-b80e-e96b44b87747 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:18.339198 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1351/2674] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:22:18 2022] POST /identity/v3/users => generated 292 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:22:18.343724 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9e1970-9c4a-4a24-af27-306f92ce80e2 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:18.357470 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9e1970-9c4a-4a24-af27-306f92ce80e2 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:18.362776 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef2b2ab-99cd-456f-a5b0-06f15ec22f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:18.363903 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef2b2ab-99cd-456f-a5b0-06f15ec22f20 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:18.363903 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef2b2ab-99cd-456f-a5b0-06f15ec22f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:18.364187 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef2b2ab-99cd-456f-a5b0-06f15ec22f20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:18.365001 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef2b2ab-99cd-456f-a5b0-06f15ec22f20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:18.366945 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef2b2ab-99cd-456f-a5b0-06f15ec22f20 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:18.379198 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1324/2675] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:22:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:22:18.383661 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48519d23-acef-4c42-b80e-e96b44b87747 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:18.394242 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48519d23-acef-4c42-b80e-e96b44b87747 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:18.399826 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9941d5-94e2-499f-8d6d-2d2804312852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:18.400569 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9941d5-94e2-499f-8d6d-2d2804312852 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:18.400860 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9941d5-94e2-499f-8d6d-2d2804312852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:18.401144 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9941d5-94e2-499f-8d6d-2d2804312852 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:18.401870 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9941d5-94e2-499f-8d6d-2d2804312852 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:18.403500 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9941d5-94e2-499f-8d6d-2d2804312852 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:18.414590 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1352/2676] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:22:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:22:18.418819 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef2b2ab-99cd-456f-a5b0-06f15ec22f20 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:18.430011 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef2b2ab-99cd-456f-a5b0-06f15ec22f20 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:18.435016 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfbe201-ed36-4db2-af93-f50f6c278b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:18.436418 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfbe201-ed36-4db2-af93-f50f6c278b00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:18.436925 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfbe201-ed36-4db2-af93-f50f6c278b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:18.437431 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfbe201-ed36-4db2-af93-f50f6c278b00 admin admin] PATH_INFO: `/v3/projects/f1676286f020448d90da478855b4bc95/users/757eae9a72ba48bf996a618ff17dfcdb/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:18.455993 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfbe201-ed36-4db2-af93-f50f6c278b00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1676286f020448d90da478855b4bc95, user_id=757eae9a72ba48bf996a618ff17dfcdb, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:18.458832 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfbe201-ed36-4db2-af93-f50f6c278b00 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:18.473913 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1325/2677] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:22:18 2022] PUT /identity/v3/projects/f1676286f020448d90da478855b4bc95/users/757eae9a72ba48bf996a618ff17dfcdb/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:22:18.484565 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b32dbf-4376-4cba-a26e-13e8ba475bc6 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:18.484887 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b32dbf-4376-4cba-a26e-13e8ba475bc6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:18.485171 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b32dbf-4376-4cba-a26e-13e8ba475bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:18.516118 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b1b32dbf-4376-4cba-a26e-13e8ba475bc6 None None] MFA Rules not processed for user `757eae9a72ba48bf996a618ff17dfcdb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:22:18.517580 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b1b32dbf-4376-4cba-a26e-13e8ba475bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:18.532176 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:22:18.593932 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1353/2678] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:22:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:22:18.598724 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfbe201-ed36-4db2-af93-f50f6c278b00 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:18.616834 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:22:18.667921 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfbe201-ed36-4db2-af93-f50f6c278b00 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:18.673124 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06532629-519f-4e4f-ba85-4c6320b7fb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:18.673840 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06532629-519f-4e4f-ba85-4c6320b7fb2b admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:18.674166 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06532629-519f-4e4f-ba85-4c6320b7fb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:18.674472 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06532629-519f-4e4f-ba85-4c6320b7fb2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:18.675315 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06532629-519f-4e4f-ba85-4c6320b7fb2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:18.677414 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06532629-519f-4e4f-ba85-4c6320b7fb2b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:18.705801 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1326/2679] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:22:18 2022] POST /identity/v3/projects => generated 383 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:22:18.710200 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b32dbf-4376-4cba-a26e-13e8ba475bc6 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:18.722635 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b32dbf-4376-4cba-a26e-13e8ba475bc6 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:18.779607 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab7526d-1bb2-45f2-92ac-9ebb03f11eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:18.780218 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7526d-1bb2-45f2-92ac-9ebb03f11eec admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:18.780502 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7526d-1bb2-45f2-92ac-9ebb03f11eec admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:18.780781 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab7526d-1bb2-45f2-92ac-9ebb03f11eec admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:18.781465 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab7526d-1bb2-45f2-92ac-9ebb03f11eec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:18.783404 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab7526d-1bb2-45f2-92ac-9ebb03f11eec admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:18.805138 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1354/2680] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:22:18 2022] POST /identity/v3/users => generated 298 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:22:18.809177 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06532629-519f-4e4f-ba85-4c6320b7fb2b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:18.819873 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06532629-519f-4e4f-ba85-4c6320b7fb2b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:18.824566 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be79d277-2c49-4ad9-885d-8d58a629121d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:18.825224 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be79d277-2c49-4ad9-885d-8d58a629121d admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:18.825510 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be79d277-2c49-4ad9-885d-8d58a629121d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:18.825785 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be79d277-2c49-4ad9-885d-8d58a629121d admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:18.826504 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be79d277-2c49-4ad9-885d-8d58a629121d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:18.828173 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be79d277-2c49-4ad9-885d-8d58a629121d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:18.839020 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1327/2681] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:22:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:22:18.843036 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab7526d-1bb2-45f2-92ac-9ebb03f11eec admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:18.853648 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab7526d-1bb2-45f2-92ac-9ebb03f11eec admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:18.858268 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf350d44-c1df-4945-af6e-c0c8437a477b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:18.859182 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf350d44-c1df-4945-af6e-c0c8437a477b admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:18.859464 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf350d44-c1df-4945-af6e-c0c8437a477b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:18.859740 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf350d44-c1df-4945-af6e-c0c8437a477b admin admin] PATH_INFO: `/v3/projects/901ad18389984af982973f7ccc3abac5/users/51dd63bbfc6e4cceb3567d04cc3e7e54/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:18.875051 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf350d44-c1df-4945-af6e-c0c8437a477b admin admin] RBAC: Authorizing `identity:create_grant(project_id=901ad18389984af982973f7ccc3abac5, user_id=51dd63bbfc6e4cceb3567d04cc3e7e54, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:18.877669 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf350d44-c1df-4945-af6e-c0c8437a477b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:18.890422 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1355/2682] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:22:18 2022] PUT /identity/v3/projects/901ad18389984af982973f7ccc3abac5/users/51dd63bbfc6e4cceb3567d04cc3e7e54/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:22:18.894418 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be79d277-2c49-4ad9-885d-8d58a629121d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:18.904729 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be79d277-2c49-4ad9-885d-8d58a629121d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:18.962583 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a0f74d-f91f-41a8-8d05-e6edca61ce5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:18.963209 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a0f74d-f91f-41a8-8d05-e6edca61ce5d admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:18.963493 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a0f74d-f91f-41a8-8d05-e6edca61ce5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:18.963765 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a0f74d-f91f-41a8-8d05-e6edca61ce5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:18.964478 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a0f74d-f91f-41a8-8d05-e6edca61ce5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:18.966056 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a0f74d-f91f-41a8-8d05-e6edca61ce5d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:18.976924 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1328/2683] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:22:18 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:22:18.981222 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf350d44-c1df-4945-af6e-c0c8437a477b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:18.992937 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf350d44-c1df-4945-af6e-c0c8437a477b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:18.997545 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed142cfb-a230-4632-aac9-6adda2fd468a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:18.998421 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed142cfb-a230-4632-aac9-6adda2fd468a admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:18.998702 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed142cfb-a230-4632-aac9-6adda2fd468a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:18.999020 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed142cfb-a230-4632-aac9-6adda2fd468a admin admin] PATH_INFO: `/v3/projects/901ad18389984af982973f7ccc3abac5/users/51dd63bbfc6e4cceb3567d04cc3e7e54/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:19.000953 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed142cfb-a230-4632-aac9-6adda2fd468a admin admin] RBAC: Authorizing `identity:create_grant(project_id=901ad18389984af982973f7ccc3abac5, user_id=51dd63bbfc6e4cceb3567d04cc3e7e54, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:19.003601 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed142cfb-a230-4632-aac9-6adda2fd468a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:19.014406 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1356/2684] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:22:18 2022] PUT /identity/v3/projects/901ad18389984af982973f7ccc3abac5/users/51dd63bbfc6e4cceb3567d04cc3e7e54/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:22:19.025846 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f776977-97d0-47d3-89c6-9b0323f613b9 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:19.026140 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f776977-97d0-47d3-89c6-9b0323f613b9 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:19.026419 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f776977-97d0-47d3-89c6-9b0323f613b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:19.053290 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3f776977-97d0-47d3-89c6-9b0323f613b9 None None] MFA Rules not processed for user `51dd63bbfc6e4cceb3567d04cc3e7e54`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:22:19.054627 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3f776977-97d0-47d3-89c6-9b0323f613b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:19.124909 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1329/2685] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:22:19 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:22:19.137101 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed142cfb-a230-4632-aac9-6adda2fd468a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:19.139057 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ed142cfb-a230-4632-aac9-6adda2fd468a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:19.201046 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed142cfb-a230-4632-aac9-6adda2fd468a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:19.204740 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203a35bd-4a7b-434f-8d14-41d615ad7b4e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:19.205387 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a35bd-4a7b-434f-8d14-41d615ad7b4e service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:19.205678 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a35bd-4a7b-434f-8d14-41d615ad7b4e service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:19.205955 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a35bd-4a7b-434f-8d14-41d615ad7b4e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:19.215660 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203a35bd-4a7b-434f-8d14-41d615ad7b4e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:19.217017 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203a35bd-4a7b-434f-8d14-41d615ad7b4e service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:19.227392 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1357/2686] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:22:19 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:22:19.315298 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f776977-97d0-47d3-89c6-9b0323f613b9 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:19.324980 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f776977-97d0-47d3-89c6-9b0323f613b9 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:19.328633 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104a0c6-ef15-4e0a-a639-6b060fae4d62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:19.329268 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104a0c6-ef15-4e0a-a639-6b060fae4d62 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:19.329550 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104a0c6-ef15-4e0a-a639-6b060fae4d62 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:19.329831 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104a0c6-ef15-4e0a-a639-6b060fae4d62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:19.338785 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:22:19.390922 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2104a0c6-ef15-4e0a-a639-6b060fae4d62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:19.392378 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2104a0c6-ef15-4e0a-a639-6b060fae4d62 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:19.401788 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1330/2687] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:22:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:22:19.687933 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203a35bd-4a7b-434f-8d14-41d615ad7b4e service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:19.698106 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:22:19.761035 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203a35bd-4a7b-434f-8d14-41d615ad7b4e service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:19.765571 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92729343-b2b1-44a3-9b69-ac67bf55b4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:19.766443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92729343-b2b1-44a3-9b69-ac67bf55b4ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:19.766727 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92729343-b2b1-44a3-9b69-ac67bf55b4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:19.767105 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92729343-b2b1-44a3-9b69-ac67bf55b4ba admin admin] PATH_INFO: `/v3/users/757eae9a72ba48bf996a618ff17dfcdb` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:19.768290 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92729343-b2b1-44a3-9b69-ac67bf55b4ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=757eae9a72ba48bf996a618ff17dfcdb)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:19.770218 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92729343-b2b1-44a3-9b69-ac67bf55b4ba admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:19.816816 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-92729343-b2b1-44a3-9b69-ac67bf55b4ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '757eae9a72ba48bf996a618ff17dfcdb'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:19.823967 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-92729343-b2b1-44a3-9b69-ac67bf55b4ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '757eae9a72ba48bf996a618ff17dfcdb'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:19.824608 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-92729343-b2b1-44a3-9b69-ac67bf55b4ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '757eae9a72ba48bf996a618ff17dfcdb'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:19.831685 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-92729343-b2b1-44a3-9b69-ac67bf55b4ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '757eae9a72ba48bf996a618ff17dfcdb'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:19.832290 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-92729343-b2b1-44a3-9b69-ac67bf55b4ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '757eae9a72ba48bf996a618ff17dfcdb'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:19.849818 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1358/2688] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:22:19 2022] DELETE /identity/v3/users/757eae9a72ba48bf996a618ff17dfcdb => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:22:19.855247 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd5f90-b33b-45b8-b531-5698023b209e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:19.855554 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd5f90-b33b-45b8-b531-5698023b209e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:19.855814 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd5f90-b33b-45b8-b531-5698023b209e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:19.873363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-0ddd5f90-b33b-45b8-b531-5698023b209e None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:22:19.874616 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0ddd5f90-b33b-45b8-b531-5698023b209e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:19.932850 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1331/2689] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:22:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:22:19.942608 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92729343-b2b1-44a3-9b69-ac67bf55b4ba admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:19.944152 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-92729343-b2b1-44a3-9b69-ac67bf55b4ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:20.003884 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92729343-b2b1-44a3-9b69-ac67bf55b4ba admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:20.007573 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07cf75a-3285-4d0e-917f-09b764c73e59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:20.008220 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07cf75a-3285-4d0e-917f-09b764c73e59 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:20.008503 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07cf75a-3285-4d0e-917f-09b764c73e59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:20.008773 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07cf75a-3285-4d0e-917f-09b764c73e59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:20.017894 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07cf75a-3285-4d0e-917f-09b764c73e59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:20.019314 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07cf75a-3285-4d0e-917f-09b764c73e59 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:20.029236 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1359/2690] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:22:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:22:20.304726 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd5f90-b33b-45b8-b531-5698023b209e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:20.306479 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-0ddd5f90-b33b-45b8-b531-5698023b209e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:20.320463 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd5f90-b33b-45b8-b531-5698023b209e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:20.325595 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa438483-473e-4705-8633-279eab80eaa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:20.326703 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa438483-473e-4705-8633-279eab80eaa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:20.327068 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa438483-473e-4705-8633-279eab80eaa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:20.327352 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa438483-473e-4705-8633-279eab80eaa0 admin admin] PATH_INFO: `/v3/projects/f1676286f020448d90da478855b4bc95` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:20.328725 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa438483-473e-4705-8633-279eab80eaa0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1676286f020448d90da478855b4bc95)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:20.331101 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa438483-473e-4705-8633-279eab80eaa0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:20.379318 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fa438483-473e-4705-8633-279eab80eaa0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1676286f020448d90da478855b4bc95', 'request_id': 'req-fa438483-473e-4705-8633-279eab80eaa0'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:20.385062 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fa438483-473e-4705-8633-279eab80eaa0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1676286f020448d90da478855b4bc95', 'request_id': 'req-fa438483-473e-4705-8633-279eab80eaa0'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:20.391192 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fa438483-473e-4705-8633-279eab80eaa0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1676286f020448d90da478855b4bc95', 'request_id': 'req-fa438483-473e-4705-8633-279eab80eaa0'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:20.403875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fa438483-473e-4705-8633-279eab80eaa0 admin admin] The token cache is being invalidate because project f1676286f020448d90da478855b4bc95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:22:20.404179 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fa438483-473e-4705-8633-279eab80eaa0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:20.404785 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-fa438483-473e-4705-8633-279eab80eaa0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:20.406336 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1332/2691] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:22:20 2022] DELETE /identity/v3/projects/f1676286f020448d90da478855b4bc95 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:22:20.410472 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07cf75a-3285-4d0e-917f-09b764c73e59 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:20.420904 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07cf75a-3285-4d0e-917f-09b764c73e59 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:20.447073 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:22:20.480378 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c30cfa4-13c0-40b3-92f1-53ac8975bbfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:20.481309 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c30cfa4-13c0-40b3-92f1-53ac8975bbfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:20.481614 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c30cfa4-13c0-40b3-92f1-53ac8975bbfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:20.481924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c30cfa4-13c0-40b3-92f1-53ac8975bbfa admin admin] PATH_INFO: `/v3/users/51dd63bbfc6e4cceb3567d04cc3e7e54` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:20.483110 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c30cfa4-13c0-40b3-92f1-53ac8975bbfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=51dd63bbfc6e4cceb3567d04cc3e7e54)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:20.485008 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c30cfa4-13c0-40b3-92f1-53ac8975bbfa admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:20.533337 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8c30cfa4-13c0-40b3-92f1-53ac8975bbfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51dd63bbfc6e4cceb3567d04cc3e7e54'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:20.540745 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8c30cfa4-13c0-40b3-92f1-53ac8975bbfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51dd63bbfc6e4cceb3567d04cc3e7e54'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:20.541418 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8c30cfa4-13c0-40b3-92f1-53ac8975bbfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51dd63bbfc6e4cceb3567d04cc3e7e54'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:20.548785 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8c30cfa4-13c0-40b3-92f1-53ac8975bbfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51dd63bbfc6e4cceb3567d04cc3e7e54'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:20.549361 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8c30cfa4-13c0-40b3-92f1-53ac8975bbfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51dd63bbfc6e4cceb3567d04cc3e7e54'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:20.567065 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1360/2692] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:22:20 2022] DELETE /identity/v3/users/51dd63bbfc6e4cceb3567d04cc3e7e54 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:22:20.839116 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa438483-473e-4705-8633-279eab80eaa0 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:20.840671 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fa438483-473e-4705-8633-279eab80eaa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:20.857880 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa438483-473e-4705-8633-279eab80eaa0 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:20.890614 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:22:20.918752 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a13f27b-216d-449d-93fb-1541d09f4bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:20.919643 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a13f27b-216d-449d-93fb-1541d09f4bbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:20.919926 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a13f27b-216d-449d-93fb-1541d09f4bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:20.920200 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a13f27b-216d-449d-93fb-1541d09f4bbe admin admin] PATH_INFO: `/v3/projects/901ad18389984af982973f7ccc3abac5` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:20.921380 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a13f27b-216d-449d-93fb-1541d09f4bbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=901ad18389984af982973f7ccc3abac5)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:20.923335 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a13f27b-216d-449d-93fb-1541d09f4bbe admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:20.970352 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0a13f27b-216d-449d-93fb-1541d09f4bbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '901ad18389984af982973f7ccc3abac5', 'request_id': 'req-0a13f27b-216d-449d-93fb-1541d09f4bbe'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:20.976628 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0a13f27b-216d-449d-93fb-1541d09f4bbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '901ad18389984af982973f7ccc3abac5', 'request_id': 'req-0a13f27b-216d-449d-93fb-1541d09f4bbe'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:20.982741 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0a13f27b-216d-449d-93fb-1541d09f4bbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '901ad18389984af982973f7ccc3abac5', 'request_id': 'req-0a13f27b-216d-449d-93fb-1541d09f4bbe'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:20.997543 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0a13f27b-216d-449d-93fb-1541d09f4bbe admin admin] The token cache is being invalidate because project 901ad18389984af982973f7ccc3abac5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:22:20.997856 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0a13f27b-216d-449d-93fb-1541d09f4bbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:20.998509 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0a13f27b-216d-449d-93fb-1541d09f4bbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:21.000143 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1333/2693] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:22:20 2022] DELETE /identity/v3/projects/901ad18389984af982973f7ccc3abac5 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:22:21.007263 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dbafac-5dd6-4ced-854f-66b5cacafccc None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:21.007875 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dbafac-5dd6-4ced-854f-66b5cacafccc None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:21.008525 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dbafac-5dd6-4ced-854f-66b5cacafccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:21.029343 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e6dbafac-5dd6-4ced-854f-66b5cacafccc None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:22:21.030548 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e6dbafac-5dd6-4ced-854f-66b5cacafccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:21.090746 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1361/2694] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:22:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:22:21.099225 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f46d09-90d6-47c1-a477-9c363c1dc782 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:21.099556 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f46d09-90d6-47c1-a477-9c363c1dc782 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:21.099900 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f46d09-90d6-47c1-a477-9c363c1dc782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:21.120316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-17f46d09-90d6-47c1-a477-9c363c1dc782 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:22:21.121583 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-17f46d09-90d6-47c1-a477-9c363c1dc782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:21.129484 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1334/2695] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:22:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:22:21.133869 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dbafac-5dd6-4ced-854f-66b5cacafccc None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:21.143331 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dbafac-5dd6-4ced-854f-66b5cacafccc None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:21.147893 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d998117a-737d-430b-a3a0-30d7da557551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:21.148578 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d998117a-737d-430b-a3a0-30d7da557551 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:21.148829 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d998117a-737d-430b-a3a0-30d7da557551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:21.149108 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d998117a-737d-430b-a3a0-30d7da557551 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:21.149835 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d998117a-737d-430b-a3a0-30d7da557551 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:22:21.150136 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d998117a-737d-430b-a3a0-30d7da557551 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:21.151815 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d998117a-737d-430b-a3a0-30d7da557551 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:21.164766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1362/2696] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:22:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:22:21.169175 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f46d09-90d6-47c1-a477-9c363c1dc782 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:21.179196 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f46d09-90d6-47c1-a477-9c363c1dc782 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:21.184076 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de034c4-3507-4923-be94-6301e850ccdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:21.184718 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de034c4-3507-4923-be94-6301e850ccdd admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:21.184997 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de034c4-3507-4923-be94-6301e850ccdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:21.185271 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de034c4-3507-4923-be94-6301e850ccdd admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:21.186005 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de034c4-3507-4923-be94-6301e850ccdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:21.188042 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de034c4-3507-4923-be94-6301e850ccdd admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:21.213395 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1335/2697] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:22:21 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:22:21.217828 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d998117a-737d-430b-a3a0-30d7da557551 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:21.228633 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d998117a-737d-430b-a3a0-30d7da557551 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:21.261288 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:22:21.289168 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4106d3d8-44c6-41a7-9dd4-2a20f742d762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:21.289838 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4106d3d8-44c6-41a7-9dd4-2a20f742d762 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:21.290121 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4106d3d8-44c6-41a7-9dd4-2a20f742d762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:21.290397 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4106d3d8-44c6-41a7-9dd4-2a20f742d762 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:21.291131 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4106d3d8-44c6-41a7-9dd4-2a20f742d762 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:21.293189 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4106d3d8-44c6-41a7-9dd4-2a20f742d762 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:21.313532 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1363/2698] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:22:21 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:22:21.317768 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de034c4-3507-4923-be94-6301e850ccdd admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:21.328093 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de034c4-3507-4923-be94-6301e850ccdd admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:21.332813 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02db7e6c-9820-4601-a678-884cf6ba75ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:21.333464 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db7e6c-9820-4601-a678-884cf6ba75ac admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:21.333746 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db7e6c-9820-4601-a678-884cf6ba75ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:21.334021 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db7e6c-9820-4601-a678-884cf6ba75ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:21.335526 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02db7e6c-9820-4601-a678-884cf6ba75ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:21.337155 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02db7e6c-9820-4601-a678-884cf6ba75ac admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:21.347866 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1336/2699] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:22:21.352775 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4106d3d8-44c6-41a7-9dd4-2a20f742d762 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:21.365574 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4106d3d8-44c6-41a7-9dd4-2a20f742d762 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:21.371918 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0ddada-fae0-494c-9f6a-6aebf5fa45a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:21.372640 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ddada-fae0-494c-9f6a-6aebf5fa45a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:21.372932 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ddada-fae0-494c-9f6a-6aebf5fa45a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:21.373215 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ddada-fae0-494c-9f6a-6aebf5fa45a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:21.373936 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0ddada-fae0-494c-9f6a-6aebf5fa45a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:21.375556 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0ddada-fae0-494c-9f6a-6aebf5fa45a5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:21.386346 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1364/2700] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:22:21.390544 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02db7e6c-9820-4601-a678-884cf6ba75ac admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:21.401476 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02db7e6c-9820-4601-a678-884cf6ba75ac admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:21.406084 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab44cc8-88d1-4e1f-a2fb-fd9d4d907568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:21.406973 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab44cc8-88d1-4e1f-a2fb-fd9d4d907568 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:21.407256 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab44cc8-88d1-4e1f-a2fb-fd9d4d907568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:21.407532 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab44cc8-88d1-4e1f-a2fb-fd9d4d907568 admin admin] PATH_INFO: `/v3/projects/2e49d5377a7c4f7db501b9f6bfa815b4/users/939a3b74559d4e7a88f68483dec56dc1/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:21.422740 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab44cc8-88d1-4e1f-a2fb-fd9d4d907568 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e49d5377a7c4f7db501b9f6bfa815b4, user_id=939a3b74559d4e7a88f68483dec56dc1, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:21.425453 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab44cc8-88d1-4e1f-a2fb-fd9d4d907568 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:21.438686 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1337/2701] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:22:21 2022] PUT /identity/v3/projects/2e49d5377a7c4f7db501b9f6bfa815b4/users/939a3b74559d4e7a88f68483dec56dc1/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:22:21.449126 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb772fae-742c-494c-ab18-6a761ea5c5c5 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:21.449693 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb772fae-742c-494c-ab18-6a761ea5c5c5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:21.449693 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb772fae-742c-494c-ab18-6a761ea5c5c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:21.477362 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-fb772fae-742c-494c-ab18-6a761ea5c5c5 None None] MFA Rules not processed for user `939a3b74559d4e7a88f68483dec56dc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:22:21.478560 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fb772fae-742c-494c-ab18-6a761ea5c5c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:21.542261 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1365/2702] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:22:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:22:21.553229 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab44cc8-88d1-4e1f-a2fb-fd9d4d907568 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:21.556492 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-7ab44cc8-88d1-4e1f-a2fb-fd9d4d907568 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:21.615130 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab44cc8-88d1-4e1f-a2fb-fd9d4d907568 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:21.619516 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58626f87-acf3-4229-a691-f4f8faaf9d94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:21.620155 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58626f87-acf3-4229-a691-f4f8faaf9d94 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:21.620436 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58626f87-acf3-4229-a691-f4f8faaf9d94 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:21.620711 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58626f87-acf3-4229-a691-f4f8faaf9d94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:21.629057 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58626f87-acf3-4229-a691-f4f8faaf9d94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:21.630511 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58626f87-acf3-4229-a691-f4f8faaf9d94 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:21.640396 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1338/2703] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:22:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:22:21.663732 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb772fae-742c-494c-ab18-6a761ea5c5c5 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:21.726983 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb772fae-742c-494c-ab18-6a761ea5c5c5 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:21.731491 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd383a4-4d86-4365-b409-461aad778757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:21.732402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd383a4-4d86-4365-b409-461aad778757 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:21.732682 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd383a4-4d86-4365-b409-461aad778757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:21.732961 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd383a4-4d86-4365-b409-461aad778757 admin admin] PATH_INFO: `/v3/users/939a3b74559d4e7a88f68483dec56dc1` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:21.734147 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd383a4-4d86-4365-b409-461aad778757 admin admin] RBAC: Authorizing `identity:delete_user(user_id=939a3b74559d4e7a88f68483dec56dc1)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:21.736086 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd383a4-4d86-4365-b409-461aad778757 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:21.781081 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:22:21.782811 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9fd383a4-4d86-4365-b409-461aad778757 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '939a3b74559d4e7a88f68483dec56dc1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:21.792941 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9fd383a4-4d86-4365-b409-461aad778757 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '939a3b74559d4e7a88f68483dec56dc1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:21.793659 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9fd383a4-4d86-4365-b409-461aad778757 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '939a3b74559d4e7a88f68483dec56dc1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:21.800808 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9fd383a4-4d86-4365-b409-461aad778757 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '939a3b74559d4e7a88f68483dec56dc1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:21.801470 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9fd383a4-4d86-4365-b409-461aad778757 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '939a3b74559d4e7a88f68483dec56dc1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:21.819280 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1366/2704] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:22:21 2022] DELETE /identity/v3/users/939a3b74559d4e7a88f68483dec56dc1 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:22:21.823898 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d308169-0be2-468a-bf5b-01fb97a5c79d None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:21.824193 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d308169-0be2-468a-bf5b-01fb97a5c79d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:21.824477 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d308169-0be2-468a-bf5b-01fb97a5c79d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:21.830261 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:22:21.844200 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-9d308169-0be2-468a-bf5b-01fb97a5c79d None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:22:21.845681 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9d308169-0be2-468a-bf5b-01fb97a5c79d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:21.904862 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1339/2705] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:22:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:22:21.915974 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd383a4-4d86-4365-b409-461aad778757 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:21.917494 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9fd383a4-4d86-4365-b409-461aad778757 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:21.975995 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd383a4-4d86-4365-b409-461aad778757 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:21.979775 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1867a-9821-4df6-876b-9bee9e3d9fb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:21.980425 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1867a-9821-4df6-876b-9bee9e3d9fb4 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:21.980707 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1867a-9821-4df6-876b-9bee9e3d9fb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:21.980982 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1867a-9821-4df6-876b-9bee9e3d9fb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:21.993196 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a1867a-9821-4df6-876b-9bee9e3d9fb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:21.994638 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a1867a-9821-4df6-876b-9bee9e3d9fb4 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:22.004135 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1367/2706] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:22:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:22:22.275381 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d308169-0be2-468a-bf5b-01fb97a5c79d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:22.276934 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9d308169-0be2-468a-bf5b-01fb97a5c79d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:22.291095 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d308169-0be2-468a-bf5b-01fb97a5c79d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:22.295924 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30ed823-42c7-4ea9-9bff-0eb61735abd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:22.296918 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ed823-42c7-4ea9-9bff-0eb61735abd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:22.297237 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ed823-42c7-4ea9-9bff-0eb61735abd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:22.297536 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ed823-42c7-4ea9-9bff-0eb61735abd7 admin admin] PATH_INFO: `/v3/projects/2e49d5377a7c4f7db501b9f6bfa815b4` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:22.298886 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30ed823-42c7-4ea9-9bff-0eb61735abd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e49d5377a7c4f7db501b9f6bfa815b4)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:22.300872 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30ed823-42c7-4ea9-9bff-0eb61735abd7 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:22.350929 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e30ed823-42c7-4ea9-9bff-0eb61735abd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e49d5377a7c4f7db501b9f6bfa815b4', 'request_id': 'req-e30ed823-42c7-4ea9-9bff-0eb61735abd7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:22.356495 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e30ed823-42c7-4ea9-9bff-0eb61735abd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e49d5377a7c4f7db501b9f6bfa815b4', 'request_id': 'req-e30ed823-42c7-4ea9-9bff-0eb61735abd7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:22.362983 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e30ed823-42c7-4ea9-9bff-0eb61735abd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e49d5377a7c4f7db501b9f6bfa815b4', 'request_id': 'req-e30ed823-42c7-4ea9-9bff-0eb61735abd7'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:22.375892 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e30ed823-42c7-4ea9-9bff-0eb61735abd7 admin admin] The token cache is being invalidate because project 2e49d5377a7c4f7db501b9f6bfa815b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:22:22.376268 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e30ed823-42c7-4ea9-9bff-0eb61735abd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:22.376964 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e30ed823-42c7-4ea9-9bff-0eb61735abd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:22.378811 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1340/2707] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:22:22 2022] DELETE /identity/v3/projects/2e49d5377a7c4f7db501b9f6bfa815b4 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:22:22.384730 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe965dab-7900-4107-a423-6cc26ef973c3 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:22.385279 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe965dab-7900-4107-a423-6cc26ef973c3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:22.385783 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe965dab-7900-4107-a423-6cc26ef973c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:22.406872 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-fe965dab-7900-4107-a423-6cc26ef973c3 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:22:22.408034 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fe965dab-7900-4107-a423-6cc26ef973c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:22.476037 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1368/2708] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:22:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:22:22.483942 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea5572a-d6e8-4f26-ad2a-91dc99fa26fb None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:22.484237 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea5572a-d6e8-4f26-ad2a-91dc99fa26fb None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:22.484520 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea5572a-d6e8-4f26-ad2a-91dc99fa26fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:22.503393 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-bea5572a-d6e8-4f26-ad2a-91dc99fa26fb None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:22:22.504580 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-bea5572a-d6e8-4f26-ad2a-91dc99fa26fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:22.513075 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1341/2709] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:22:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:22:22.517261 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe965dab-7900-4107-a423-6cc26ef973c3 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:22.527530 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe965dab-7900-4107-a423-6cc26ef973c3 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:22.532027 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdc3dda-1f26-4e39-a9f0-fafd40775185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:22.532698 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc3dda-1f26-4e39-a9f0-fafd40775185 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:22.532976 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc3dda-1f26-4e39-a9f0-fafd40775185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:22.533245 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc3dda-1f26-4e39-a9f0-fafd40775185 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:22.533962 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdc3dda-1f26-4e39-a9f0-fafd40775185 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:22:22.534262 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdc3dda-1f26-4e39-a9f0-fafd40775185 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:22.535873 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdc3dda-1f26-4e39-a9f0-fafd40775185 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:22.547485 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1369/2710] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:22:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 27 15:22:22.551766 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea5572a-d6e8-4f26-ad2a-91dc99fa26fb None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:22.561392 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea5572a-d6e8-4f26-ad2a-91dc99fa26fb None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:22.566106 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c022c1-8d9d-4a6b-8407-3932680864d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:22.566751 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c022c1-8d9d-4a6b-8407-3932680864d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:22.567062 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c022c1-8d9d-4a6b-8407-3932680864d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:22.567341 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c022c1-8d9d-4a6b-8407-3932680864d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:22.568035 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c022c1-8d9d-4a6b-8407-3932680864d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:22.570503 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c022c1-8d9d-4a6b-8407-3932680864d5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:22.598718 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1342/2711] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:22:22 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:22:22.603987 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdc3dda-1f26-4e39-a9f0-fafd40775185 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:22.613988 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdc3dda-1f26-4e39-a9f0-fafd40775185 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:22.663643 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:22:22.674092 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09694537-aebe-4658-aad2-38d762faf26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:22.674776 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09694537-aebe-4658-aad2-38d762faf26f admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:22.675093 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09694537-aebe-4658-aad2-38d762faf26f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:22.675373 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09694537-aebe-4658-aad2-38d762faf26f admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:22.676067 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09694537-aebe-4658-aad2-38d762faf26f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:22.677975 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09694537-aebe-4658-aad2-38d762faf26f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:22.698312 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1370/2712] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:22:22 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:22:22.703336 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c022c1-8d9d-4a6b-8407-3932680864d5 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:22.720958 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c022c1-8d9d-4a6b-8407-3932680864d5 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:22.727064 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f9a7f1-233d-4298-84d0-18e399fbf3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:22.727744 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9a7f1-233d-4298-84d0-18e399fbf3d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:22.728030 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9a7f1-233d-4298-84d0-18e399fbf3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:22.728310 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f9a7f1-233d-4298-84d0-18e399fbf3d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:22.729037 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f9a7f1-233d-4298-84d0-18e399fbf3d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:22.730657 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f9a7f1-233d-4298-84d0-18e399fbf3d9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:22.741447 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1343/2713] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:22:22 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:22:22.745834 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09694537-aebe-4658-aad2-38d762faf26f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:22.756026 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09694537-aebe-4658-aad2-38d762faf26f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:22.761797 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d5efd-ca0b-4840-8d67-a83be8c303e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:22.762482 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d5efd-ca0b-4840-8d67-a83be8c303e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:22.762794 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d5efd-ca0b-4840-8d67-a83be8c303e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:22.763094 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d5efd-ca0b-4840-8d67-a83be8c303e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:22.763832 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5d5efd-ca0b-4840-8d67-a83be8c303e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:22.765447 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5d5efd-ca0b-4840-8d67-a83be8c303e6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:22.780308 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1371/2714] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:22:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:22:22.785981 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f9a7f1-233d-4298-84d0-18e399fbf3d9 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:22.816087 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f9a7f1-233d-4298-84d0-18e399fbf3d9 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:22.822016 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0520d25-d784-4944-ac3e-9acab8656215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:22.822944 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0520d25-d784-4944-ac3e-9acab8656215 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:22.823256 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0520d25-d784-4944-ac3e-9acab8656215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:22.823535 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0520d25-d784-4944-ac3e-9acab8656215 admin admin] PATH_INFO: `/v3/projects/651244a3109d4dfe9b8317dc2f721eaa/users/1d2d6556190b4e9cae3cb01b63a5add9/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:22.839908 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0520d25-d784-4944-ac3e-9acab8656215 admin admin] RBAC: Authorizing `identity:create_grant(project_id=651244a3109d4dfe9b8317dc2f721eaa, user_id=1d2d6556190b4e9cae3cb01b63a5add9, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:22.844969 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0520d25-d784-4944-ac3e-9acab8656215 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:22.860095 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1344/2715] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:22:22 2022] PUT /identity/v3/projects/651244a3109d4dfe9b8317dc2f721eaa/users/1d2d6556190b4e9cae3cb01b63a5add9/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:22:22.871035 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf707d0-bd07-4c81-a9fe-686cb06b44f8 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:22.871328 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf707d0-bd07-4c81-a9fe-686cb06b44f8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:22.871610 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf707d0-bd07-4c81-a9fe-686cb06b44f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:22.899037 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6bf707d0-bd07-4c81-a9fe-686cb06b44f8 None None] MFA Rules not processed for user `1d2d6556190b4e9cae3cb01b63a5add9`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:22:22.900269 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6bf707d0-bd07-4c81-a9fe-686cb06b44f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:22.973836 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1372/2716] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:22:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:22:22.990224 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0520d25-d784-4944-ac3e-9acab8656215 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:22.992769 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c0520d25-d784-4944-ac3e-9acab8656215 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:23.032203 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:22:23.082162 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0520d25-d784-4944-ac3e-9acab8656215 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:23.086634 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ed1317-83a1-4d7d-8b2b-2a60161a337c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:23.087362 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed1317-83a1-4d7d-8b2b-2a60161a337c service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:23.087650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed1317-83a1-4d7d-8b2b-2a60161a337c service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:23.087959 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ed1317-83a1-4d7d-8b2b-2a60161a337c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:23.100722 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ed1317-83a1-4d7d-8b2b-2a60161a337c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:23.103485 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ed1317-83a1-4d7d-8b2b-2a60161a337c service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:23.120424 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1345/2717] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:22:22 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:22:23.151160 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf707d0-bd07-4c81-a9fe-686cb06b44f8 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:23.242982 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf707d0-bd07-4c81-a9fe-686cb06b44f8 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:23.247712 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b021d0-e102-4cf9-8f64-9c6d45580cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:23.248589 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b021d0-e102-4cf9-8f64-9c6d45580cf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:23.248927 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b021d0-e102-4cf9-8f64-9c6d45580cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:23.249208 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b021d0-e102-4cf9-8f64-9c6d45580cf8 admin admin] PATH_INFO: `/v3/users/1d2d6556190b4e9cae3cb01b63a5add9` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:23.250414 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b021d0-e102-4cf9-8f64-9c6d45580cf8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d2d6556190b4e9cae3cb01b63a5add9)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:23.253372 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b021d0-e102-4cf9-8f64-9c6d45580cf8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:23.316768 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-13b021d0-e102-4cf9-8f64-9c6d45580cf8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d2d6556190b4e9cae3cb01b63a5add9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:23.325796 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-13b021d0-e102-4cf9-8f64-9c6d45580cf8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d2d6556190b4e9cae3cb01b63a5add9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:23.326573 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-13b021d0-e102-4cf9-8f64-9c6d45580cf8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d2d6556190b4e9cae3cb01b63a5add9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:23.335795 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-13b021d0-e102-4cf9-8f64-9c6d45580cf8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d2d6556190b4e9cae3cb01b63a5add9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:23.336563 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-13b021d0-e102-4cf9-8f64-9c6d45580cf8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d2d6556190b4e9cae3cb01b63a5add9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:23.338092 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:22:23.360577 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1373/2718] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:22:23 2022] DELETE /identity/v3/users/1d2d6556190b4e9cae3cb01b63a5add9 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:22:23.365273 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9255d77d-3afc-432a-a8eb-8368364a5e72 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:23.365804 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9255d77d-3afc-432a-a8eb-8368364a5e72 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:23.365934 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9255d77d-3afc-432a-a8eb-8368364a5e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:23.387215 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-9255d77d-3afc-432a-a8eb-8368364a5e72 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:22:23.389103 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9255d77d-3afc-432a-a8eb-8368364a5e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:23.457696 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1346/2719] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:22:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:22:23.468133 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b021d0-e102-4cf9-8f64-9c6d45580cf8 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:23.469901 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-13b021d0-e102-4cf9-8f64-9c6d45580cf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:23.537002 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b021d0-e102-4cf9-8f64-9c6d45580cf8 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:23.541652 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749719ae-cb64-412e-8e1e-e8a7a7137ab6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:23.542306 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749719ae-cb64-412e-8e1e-e8a7a7137ab6 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:23.542592 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749719ae-cb64-412e-8e1e-e8a7a7137ab6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:23.542962 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749719ae-cb64-412e-8e1e-e8a7a7137ab6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:23.552693 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749719ae-cb64-412e-8e1e-e8a7a7137ab6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:23.554245 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749719ae-cb64-412e-8e1e-e8a7a7137ab6 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:23.564976 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1374/2720] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:22:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:22:23.885515 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9255d77d-3afc-432a-a8eb-8368364a5e72 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:23.887325 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9255d77d-3afc-432a-a8eb-8368364a5e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:23.902726 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9255d77d-3afc-432a-a8eb-8368364a5e72 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:23.908055 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640878b-608c-42ce-bcb7-6a9630b59a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:23.908937 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640878b-608c-42ce-bcb7-6a9630b59a6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:23.909227 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640878b-608c-42ce-bcb7-6a9630b59a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:23.909502 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640878b-608c-42ce-bcb7-6a9630b59a6d admin admin] PATH_INFO: `/v3/projects/651244a3109d4dfe9b8317dc2f721eaa` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:23.910831 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e640878b-608c-42ce-bcb7-6a9630b59a6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=651244a3109d4dfe9b8317dc2f721eaa)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:23.912991 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e640878b-608c-42ce-bcb7-6a9630b59a6d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:23.964359 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e640878b-608c-42ce-bcb7-6a9630b59a6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '651244a3109d4dfe9b8317dc2f721eaa', 'request_id': 'req-e640878b-608c-42ce-bcb7-6a9630b59a6d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:23.970040 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e640878b-608c-42ce-bcb7-6a9630b59a6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '651244a3109d4dfe9b8317dc2f721eaa', 'request_id': 'req-e640878b-608c-42ce-bcb7-6a9630b59a6d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:23.976751 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e640878b-608c-42ce-bcb7-6a9630b59a6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '651244a3109d4dfe9b8317dc2f721eaa', 'request_id': 'req-e640878b-608c-42ce-bcb7-6a9630b59a6d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:23.990123 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e640878b-608c-42ce-bcb7-6a9630b59a6d admin admin] The token cache is being invalidate because project 651244a3109d4dfe9b8317dc2f721eaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:22:23.990402 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e640878b-608c-42ce-bcb7-6a9630b59a6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:23.992308 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e640878b-608c-42ce-bcb7-6a9630b59a6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:23.993918 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1347/2721] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:22:23 2022] DELETE /identity/v3/projects/651244a3109d4dfe9b8317dc2f721eaa => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:22:33.560060 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=33,keystone:DELETE=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:22:33.987130 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=70,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:22:33.987739 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749719ae-cb64-412e-8e1e-e8a7a7137ab6 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:33.989915 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-749719ae-cb64-412e-8e1e-e8a7a7137ab6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:34.048821 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749719ae-cb64-412e-8e1e-e8a7a7137ab6 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:34.096842 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b26d9-2081-4bfe-89cf-9647dc1261fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:34.097488 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b26d9-2081-4bfe-89cf-9647dc1261fb service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:34.097767 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b26d9-2081-4bfe-89cf-9647dc1261fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:34.098039 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b26d9-2081-4bfe-89cf-9647dc1261fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:34.100438 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d10b26d9-2081-4bfe-89cf-9647dc1261fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:34.167093 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10b26d9-2081-4bfe-89cf-9647dc1261fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:34.168585 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10b26d9-2081-4bfe-89cf-9647dc1261fb service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:34.177778 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1375/2722] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:22:33 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:22:42.067958 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f6f7f3-7791-4ca1-bb2e-2a1f12b48865 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:42.068511 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f6f7f3-7791-4ca1-bb2e-2a1f12b48865 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:42.068617 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f6f7f3-7791-4ca1-bb2e-2a1f12b48865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:42.086711 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a5f6f7f3-7791-4ca1-bb2e-2a1f12b48865 None None] MFA Rules not processed for user `af74115f423c43fdb2544876be4e5622`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:22:42.087950 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a5f6f7f3-7791-4ca1-bb2e-2a1f12b48865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:42.095104 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1348/2723] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:22:42 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:22:44.173612 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=68 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:22:44.563281 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b26d9-2081-4bfe-89cf-9647dc1261fb service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:44.564898 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d10b26d9-2081-4bfe-89cf-9647dc1261fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:44.643155 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b26d9-2081-4bfe-89cf-9647dc1261fb service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:44.647656 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1534a4-7a3c-462e-b226-494307b38db8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:44.648653 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1534a4-7a3c-462e-b226-494307b38db8 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:44.648938 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1534a4-7a3c-462e-b226-494307b38db8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:44.649214 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1534a4-7a3c-462e-b226-494307b38db8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:44.651096 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ec1534a4-7a3c-462e-b226-494307b38db8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:44.738717 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1534a4-7a3c-462e-b226-494307b38db8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:44.740729 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1534a4-7a3c-462e-b226-494307b38db8 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:44.765854 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1376/2724] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:22:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:22:48.374474 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f6f7f3-7791-4ca1-bb2e-2a1f12b48865 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:48.376002 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a5f6f7f3-7791-4ca1-bb2e-2a1f12b48865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:48.447978 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1534a4-7a3c-462e-b226-494307b38db8 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:48.449493 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ec1534a4-7a3c-462e-b226-494307b38db8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:48.468017 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1534a4-7a3c-462e-b226-494307b38db8 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:48.474063 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dee38ef-13ec-4e12-a13a-393507d76979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:48.480490 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee38ef-13ec-4e12-a13a-393507d76979 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:48.480967 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee38ef-13ec-4e12-a13a-393507d76979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:48.481328 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dee38ef-13ec-4e12-a13a-393507d76979 admin admin] PATH_INFO: `/v3/users/5a990918de6d409a949616da90de6a5e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:48.482708 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f6f7f3-7791-4ca1-bb2e-2a1f12b48865 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:48.483541 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dee38ef-13ec-4e12-a13a-393507d76979 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a990918de6d409a949616da90de6a5e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:48.485481 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dee38ef-13ec-4e12-a13a-393507d76979 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:48.489068 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a580c311-190a-402c-a52c-562d7d9ea722 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8293dab3ffd647f5927d323572fdbdce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:48.489747 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580c311-190a-402c-a52c-562d7d9ea722 service placement] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:48.490191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580c311-190a-402c-a52c-562d7d9ea722 service placement] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:48.490472 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580c311-190a-402c-a52c-562d7d9ea722 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:48.492394 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a580c311-190a-402c-a52c-562d7d9ea722 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:48.506197 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a580c311-190a-402c-a52c-562d7d9ea722 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:48.509209 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a580c311-190a-402c-a52c-562d7d9ea722 service placement] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:48.521044 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1349/2725] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:22:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:22:48.554978 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5dee38ef-13ec-4e12-a13a-393507d76979 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a990918de6d409a949616da90de6a5e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:48.563020 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5dee38ef-13ec-4e12-a13a-393507d76979 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a990918de6d409a949616da90de6a5e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:48.563717 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5dee38ef-13ec-4e12-a13a-393507d76979 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a990918de6d409a949616da90de6a5e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:48.571172 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5dee38ef-13ec-4e12-a13a-393507d76979 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a990918de6d409a949616da90de6a5e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:48.571788 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5dee38ef-13ec-4e12-a13a-393507d76979 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a990918de6d409a949616da90de6a5e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:48.590293 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1377/2726] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:22:48 2022] DELETE /identity/v3/users/5a990918de6d409a949616da90de6a5e => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:22:48.761031 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a580c311-190a-402c-a52c-562d7d9ea722 service placement] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:48.780275 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a580c311-190a-402c-a52c-562d7d9ea722 service placement] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:48.863386 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10702e7-a011-48ef-8219-52368ab9a47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:48.864254 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10702e7-a011-48ef-8219-52368ab9a47e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:48.864545 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10702e7-a011-48ef-8219-52368ab9a47e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:48.864802 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10702e7-a011-48ef-8219-52368ab9a47e admin admin] PATH_INFO: `/v3/projects/f20b51981801495aacd31a56e87d0e3c` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:48.866088 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10702e7-a011-48ef-8219-52368ab9a47e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f20b51981801495aacd31a56e87d0e3c)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:48.868084 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10702e7-a011-48ef-8219-52368ab9a47e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:48.920143 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e10702e7-a011-48ef-8219-52368ab9a47e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f20b51981801495aacd31a56e87d0e3c', 'request_id': 'req-e10702e7-a011-48ef-8219-52368ab9a47e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:48.925724 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e10702e7-a011-48ef-8219-52368ab9a47e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f20b51981801495aacd31a56e87d0e3c', 'request_id': 'req-e10702e7-a011-48ef-8219-52368ab9a47e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:48.932072 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e10702e7-a011-48ef-8219-52368ab9a47e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f20b51981801495aacd31a56e87d0e3c', 'request_id': 'req-e10702e7-a011-48ef-8219-52368ab9a47e'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:48.944324 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e10702e7-a011-48ef-8219-52368ab9a47e admin admin] The token cache is being invalidate because project f20b51981801495aacd31a56e87d0e3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:22:48.944610 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e10702e7-a011-48ef-8219-52368ab9a47e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:48.945224 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-e10702e7-a011-48ef-8219-52368ab9a47e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:22:48.946754 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1350/2727] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:22:48 2022] DELETE /identity/v3/projects/f20b51981801495aacd31a56e87d0e3c => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:22:48.951904 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c7e34-98d2-425d-8edb-ba67e68051dd None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:48.952224 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c7e34-98d2-425d-8edb-ba67e68051dd None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:48.952500 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9c7e34-98d2-425d-8edb-ba67e68051dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:48.970885 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-bc9c7e34-98d2-425d-8edb-ba67e68051dd None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:22:48.972009 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bc9c7e34-98d2-425d-8edb-ba67e68051dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:48.989541 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:22:49.034783 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1378/2728] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:22:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:22:49.042784 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded03ca-8677-466e-a5ff-f9f3fbd175f8 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:49.043134 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded03ca-8677-466e-a5ff-f9f3fbd175f8 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:49.043415 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cded03ca-8677-466e-a5ff-f9f3fbd175f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:49.062385 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-cded03ca-8677-466e-a5ff-f9f3fbd175f8 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:22:49.063627 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-cded03ca-8677-466e-a5ff-f9f3fbd175f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:49.071978 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1351/2729] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:22:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:22:49.076146 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9c7e34-98d2-425d-8edb-ba67e68051dd None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:49.086342 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9c7e34-98d2-425d-8edb-ba67e68051dd None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:49.090796 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3655b1b5-6f7d-4ae6-ae69-3342ddb4f668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:49.091468 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655b1b5-6f7d-4ae6-ae69-3342ddb4f668 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:49.091875 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655b1b5-6f7d-4ae6-ae69-3342ddb4f668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:49.092152 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3655b1b5-6f7d-4ae6-ae69-3342ddb4f668 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:49.092867 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3655b1b5-6f7d-4ae6-ae69-3342ddb4f668 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:22:49.093170 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3655b1b5-6f7d-4ae6-ae69-3342ddb4f668 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:49.095065 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3655b1b5-6f7d-4ae6-ae69-3342ddb4f668 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:49.107196 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1379/2730] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:22:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:22:49.111539 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cded03ca-8677-466e-a5ff-f9f3fbd175f8 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:49.121630 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cded03ca-8677-466e-a5ff-f9f3fbd175f8 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:49.126344 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d263b-da58-40dc-b137-8f2a66474502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:49.127034 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d263b-da58-40dc-b137-8f2a66474502 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:49.127324 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d263b-da58-40dc-b137-8f2a66474502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:49.127602 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d263b-da58-40dc-b137-8f2a66474502 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:49.128303 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979d263b-da58-40dc-b137-8f2a66474502 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:49.130235 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979d263b-da58-40dc-b137-8f2a66474502 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:49.134471 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:22:49.157545 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1352/2731] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:22:49 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:22:49.162313 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3655b1b5-6f7d-4ae6-ae69-3342ddb4f668 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:49.172187 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3655b1b5-6f7d-4ae6-ae69-3342ddb4f668 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:49.229308 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb8cfc-59f0-4410-bffc-795c6c362513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:49.229905 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb8cfc-59f0-4410-bffc-795c6c362513 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:49.230194 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb8cfc-59f0-4410-bffc-795c6c362513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:49.230474 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb8cfc-59f0-4410-bffc-795c6c362513 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:49.231305 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cb8cfc-59f0-4410-bffc-795c6c362513 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:49.233104 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cb8cfc-59f0-4410-bffc-795c6c362513 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:49.252827 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1380/2732] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:22:49 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:22:49.257428 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d263b-da58-40dc-b137-8f2a66474502 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:49.267556 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d263b-da58-40dc-b137-8f2a66474502 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:49.272221 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce4dbb4-7ce8-4f09-a54f-93bb044ee72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:49.272873 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce4dbb4-7ce8-4f09-a54f-93bb044ee72b admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:49.273159 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce4dbb4-7ce8-4f09-a54f-93bb044ee72b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:49.273437 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce4dbb4-7ce8-4f09-a54f-93bb044ee72b admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:49.274163 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce4dbb4-7ce8-4f09-a54f-93bb044ee72b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:49.275784 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce4dbb4-7ce8-4f09-a54f-93bb044ee72b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:49.286062 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1353/2733] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:22:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:22:49.290473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb8cfc-59f0-4410-bffc-795c6c362513 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:49.300180 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb8cfc-59f0-4410-bffc-795c6c362513 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:49.305412 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3902b1-a507-4ace-afa8-859aacbaab1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:49.306057 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3902b1-a507-4ace-afa8-859aacbaab1e admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:49.306345 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3902b1-a507-4ace-afa8-859aacbaab1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:49.306627 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3902b1-a507-4ace-afa8-859aacbaab1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:49.307370 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3902b1-a507-4ace-afa8-859aacbaab1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:49.308951 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3902b1-a507-4ace-afa8-859aacbaab1e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:49.318799 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1381/2734] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:22:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:22:49.322822 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce4dbb4-7ce8-4f09-a54f-93bb044ee72b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:49.332688 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce4dbb4-7ce8-4f09-a54f-93bb044ee72b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:49.337316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dc6bab-e755-4a00-9ccf-c7c8d4c26758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:49.338234 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc6bab-e755-4a00-9ccf-c7c8d4c26758 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:49.338538 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc6bab-e755-4a00-9ccf-c7c8d4c26758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:49.338819 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dc6bab-e755-4a00-9ccf-c7c8d4c26758 admin admin] PATH_INFO: `/v3/projects/f027c3b75bc6430abd765a26269f9058/users/b1302d3b8649407ca010f12aad7a6597/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:49.354431 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dc6bab-e755-4a00-9ccf-c7c8d4c26758 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f027c3b75bc6430abd765a26269f9058, user_id=b1302d3b8649407ca010f12aad7a6597, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:49.357114 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dc6bab-e755-4a00-9ccf-c7c8d4c26758 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:49.370156 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1354/2735] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:22:49 2022] PUT /identity/v3/projects/f027c3b75bc6430abd765a26269f9058/users/b1302d3b8649407ca010f12aad7a6597/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:22:49.375208 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd20f05-94ba-40d1-9258-fe56abf067d5 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:49.375502 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd20f05-94ba-40d1-9258-fe56abf067d5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:49.375787 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd20f05-94ba-40d1-9258-fe56abf067d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:49.393093 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4dd20f05-94ba-40d1-9258-fe56abf067d5 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:22:49.394273 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4dd20f05-94ba-40d1-9258-fe56abf067d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:49.451972 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1382/2736] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:22:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:22:49.462129 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dc6bab-e755-4a00-9ccf-c7c8d4c26758 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:49.463906 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-25dc6bab-e755-4a00-9ccf-c7c8d4c26758 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:49.519873 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dc6bab-e755-4a00-9ccf-c7c8d4c26758 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:49.524164 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa52db5c-a6bc-4aaa-b37f-3315d7e53ada service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:49.524802 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa52db5c-a6bc-4aaa-b37f-3315d7e53ada service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:49.525085 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa52db5c-a6bc-4aaa-b37f-3315d7e53ada service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:49.525364 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa52db5c-a6bc-4aaa-b37f-3315d7e53ada service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:49.534203 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa52db5c-a6bc-4aaa-b37f-3315d7e53ada service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:49.535661 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa52db5c-a6bc-4aaa-b37f-3315d7e53ada service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:49.545515 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1355/2737] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:22:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:22:53.664896 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a9edc-1265-4664-a005-8228e3c52d50 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:53.665529 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a9edc-1265-4664-a005-8228e3c52d50 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:53.665529 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a9edc-1265-4664-a005-8228e3c52d50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:53.695524 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-cd0a9edc-1265-4664-a005-8228e3c52d50 None None] MFA Rules not processed for user `b1302d3b8649407ca010f12aad7a6597`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:22:53.696667 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cd0a9edc-1265-4664-a005-8228e3c52d50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:53.721130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:22:53.785695 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1383/2738] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:22:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:22:53.804035 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa52db5c-a6bc-4aaa-b37f-3315d7e53ada service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:53.806453 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fa52db5c-a6bc-4aaa-b37f-3315d7e53ada service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:53.903238 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa52db5c-a6bc-4aaa-b37f-3315d7e53ada service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:53.909192 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed42585-609d-49bc-b867-b6fba3f0ba42 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '451a9232f95e494683d81b6078a318d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:53.909851 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed42585-609d-49bc-b867-b6fba3f0ba42 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:53.910134 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed42585-609d-49bc-b867-b6fba3f0ba42 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:53.910411 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed42585-609d-49bc-b867-b6fba3f0ba42 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:53.919898 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed42585-609d-49bc-b867-b6fba3f0ba42 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:53.921620 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed42585-609d-49bc-b867-b6fba3f0ba42 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:53.932344 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1356/2739] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:22:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:22:55.647405 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31406d97-a5b7-4335-acc9-a56243ecdd44 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:55.647975 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31406d97-a5b7-4335-acc9-a56243ecdd44 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:55.648099 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31406d97-a5b7-4335-acc9-a56243ecdd44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:55.666343 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-31406d97-a5b7-4335-acc9-a56243ecdd44 None None] MFA Rules not processed for user `b1302d3b8649407ca010f12aad7a6597`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:22:55.667665 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-31406d97-a5b7-4335-acc9-a56243ecdd44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:55.681099 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1384/2740] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:22:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:22:58.962897 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed42585-609d-49bc-b867-b6fba3f0ba42 service nova] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:22:58.972855 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed42585-609d-49bc-b867-b6fba3f0ba42 service nova] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:22:58.979549 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b668a239-791a-4457-b250-c8cffc09383b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:22:58.980478 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668a239-791a-4457-b250-c8cffc09383b service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:22:58.980765 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668a239-791a-4457-b250-c8cffc09383b service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:22:58.981042 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668a239-791a-4457-b250-c8cffc09383b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:22:58.982810 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b668a239-791a-4457-b250-c8cffc09383b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:22:59.020334 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:22:59.053338 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b668a239-791a-4457-b250-c8cffc09383b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:22:59.054824 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b668a239-791a-4457-b250-c8cffc09383b service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:22:59.064778 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1357/2741] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:22:58 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:23:05.660286 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=22 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:23:09.060317 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=13 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:23:19.982535 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31406d97-a5b7-4335-acc9-a56243ecdd44 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:23:19.984051 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-31406d97-a5b7-4335-acc9-a56243ecdd44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:23:20.001292 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31406d97-a5b7-4335-acc9-a56243ecdd44 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:23:20.007198 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e3ca8-1ec7-4060-9676-6fd45a10c69c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '451a9232f95e494683d81b6078a318d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:23:20.007841 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e3ca8-1ec7-4060-9676-6fd45a10c69c service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:23:20.008126 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e3ca8-1ec7-4060-9676-6fd45a10c69c service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:23:20.008403 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e3ca8-1ec7-4060-9676-6fd45a10c69c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:23:20.010175 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-180e3ca8-1ec7-4060-9676-6fd45a10c69c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:23:20.069288 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180e3ca8-1ec7-4060-9676-6fd45a10c69c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:23:20.070910 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180e3ca8-1ec7-4060-9676-6fd45a10c69c service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:23:20.080038 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1385/2742] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:23:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:23:21.750274 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b668a239-791a-4457-b250-c8cffc09383b service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:23:21.751855 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b668a239-791a-4457-b250-c8cffc09383b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:23:21.820993 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b668a239-791a-4457-b250-c8cffc09383b service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:23:21.825998 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e34d8cb-cae1-49ea-b3f2-88c0c80fde7b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8293dab3ffd647f5927d323572fdbdce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:23:21.826640 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34d8cb-cae1-49ea-b3f2-88c0c80fde7b service placement] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:23:21.826970 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34d8cb-cae1-49ea-b3f2-88c0c80fde7b service placement] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:23:21.827256 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34d8cb-cae1-49ea-b3f2-88c0c80fde7b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:23:21.829038 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6e34d8cb-cae1-49ea-b3f2-88c0c80fde7b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:23:21.840921 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e34d8cb-cae1-49ea-b3f2-88c0c80fde7b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:23:21.842500 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e34d8cb-cae1-49ea-b3f2-88c0c80fde7b service placement] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:23:21.852624 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1358/2743] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:23:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:23:23.699621 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e3ca8-1ec7-4060-9676-6fd45a10c69c service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:23:23.701067 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-180e3ca8-1ec7-4060-9676-6fd45a10c69c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:23:23.714350 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e3ca8-1ec7-4060-9676-6fd45a10c69c service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:23:23.717947 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e9e0c5-8224-4517-9fce-0edf6bb28bcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:23:23.718516 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e9e0c5-8224-4517-9fce-0edf6bb28bcc service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:23:23.718797 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e9e0c5-8224-4517-9fce-0edf6bb28bcc service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:23:23.719114 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e9e0c5-8224-4517-9fce-0edf6bb28bcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:23:23.721115 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-51e9e0c5-8224-4517-9fce-0edf6bb28bcc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:23:23.735671 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e9e0c5-8224-4517-9fce-0edf6bb28bcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:23:23.737191 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e9e0c5-8224-4517-9fce-0edf6bb28bcc service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:23:23.748444 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1386/2744] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:23:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:23:30.136940 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e34d8cb-cae1-49ea-b3f2-88c0c80fde7b service placement] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:23:30.138402 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-6e34d8cb-cae1-49ea-b3f2-88c0c80fde7b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:23:30.152407 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e34d8cb-cae1-49ea-b3f2-88c0c80fde7b service placement] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:23:30.157185 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e72cbd-0e28-45c6-b4b9-fb357ce4122a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:23:30.158110 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e72cbd-0e28-45c6-b4b9-fb357ce4122a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:23:30.158464 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e72cbd-0e28-45c6-b4b9-fb357ce4122a admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:23:30.158805 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e72cbd-0e28-45c6-b4b9-fb357ce4122a admin admin] PATH_INFO: `/v3/users/f52a9eabb6f84931b8f0af715af9eb36` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:23:30.160037 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e72cbd-0e28-45c6-b4b9-fb357ce4122a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f52a9eabb6f84931b8f0af715af9eb36)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:23:30.162014 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e72cbd-0e28-45c6-b4b9-fb357ce4122a admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:23:30.208009 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b2e72cbd-0e28-45c6-b4b9-fb357ce4122a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f52a9eabb6f84931b8f0af715af9eb36'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:23:30.215336 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b2e72cbd-0e28-45c6-b4b9-fb357ce4122a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f52a9eabb6f84931b8f0af715af9eb36'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:23:30.222390 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b2e72cbd-0e28-45c6-b4b9-fb357ce4122a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f52a9eabb6f84931b8f0af715af9eb36'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:23:30.223057 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b2e72cbd-0e28-45c6-b4b9-fb357ce4122a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f52a9eabb6f84931b8f0af715af9eb36'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:23:30.223672 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b2e72cbd-0e28-45c6-b4b9-fb357ce4122a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f52a9eabb6f84931b8f0af715af9eb36'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:23:30.241036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1359/2745] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:23:30 2022] DELETE /identity/v3/users/f52a9eabb6f84931b8f0af715af9eb36 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:23:30.411863 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e9e0c5-8224-4517-9fce-0edf6bb28bcc service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:23:30.425303 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e9e0c5-8224-4517-9fce-0edf6bb28bcc service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:23:30.491307 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b61958-2e2c-4bd2-9495-9cb3f5750500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:23:30.492107 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b61958-2e2c-4bd2-9495-9cb3f5750500 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:23:30.492397 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b61958-2e2c-4bd2-9495-9cb3f5750500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:23:30.492852 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b61958-2e2c-4bd2-9495-9cb3f5750500 admin admin] PATH_INFO: `/v3/projects/520d6685a60d4d9b8b7415d11b01a8f0` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:23:30.494192 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b61958-2e2c-4bd2-9495-9cb3f5750500 admin admin] RBAC: Authorizing `identity:delete_project(project_id=520d6685a60d4d9b8b7415d11b01a8f0)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:23:30.496478 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b61958-2e2c-4bd2-9495-9cb3f5750500 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:23:30.545200 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f2b61958-2e2c-4bd2-9495-9cb3f5750500 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '520d6685a60d4d9b8b7415d11b01a8f0', 'request_id': 'req-f2b61958-2e2c-4bd2-9495-9cb3f5750500'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:23:30.553270 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f2b61958-2e2c-4bd2-9495-9cb3f5750500 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '520d6685a60d4d9b8b7415d11b01a8f0', 'request_id': 'req-f2b61958-2e2c-4bd2-9495-9cb3f5750500'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:23:30.558691 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f2b61958-2e2c-4bd2-9495-9cb3f5750500 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '520d6685a60d4d9b8b7415d11b01a8f0', 'request_id': 'req-f2b61958-2e2c-4bd2-9495-9cb3f5750500'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:23:30.570127 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f2b61958-2e2c-4bd2-9495-9cb3f5750500 admin admin] The token cache is being invalidate because project 520d6685a60d4d9b8b7415d11b01a8f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:23:30.570415 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f2b61958-2e2c-4bd2-9495-9cb3f5750500 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:23:30.571085 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f2b61958-2e2c-4bd2-9495-9cb3f5750500 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:23:30.572647 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1387/2746] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:23:30 2022] DELETE /identity/v3/projects/520d6685a60d4d9b8b7415d11b01a8f0 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:23:30.576577 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e72cbd-0e28-45c6-b4b9-fb357ce4122a admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:23:30.577989 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b2e72cbd-0e28-45c6-b4b9-fb357ce4122a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:23:30.591384 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e72cbd-0e28-45c6-b4b9-fb357ce4122a admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:23:30.648749 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeaff37-50d4-4f86-98f0-839905a1bbd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:23:30.649627 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeaff37-50d4-4f86-98f0-839905a1bbd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:23:30.649913 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeaff37-50d4-4f86-98f0-839905a1bbd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:23:30.650191 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeaff37-50d4-4f86-98f0-839905a1bbd4 admin admin] PATH_INFO: `/v3/users/94926e51a8954fd8a836eba351a176de` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:23:30.651398 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeaff37-50d4-4f86-98f0-839905a1bbd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94926e51a8954fd8a836eba351a176de)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:23:30.653325 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeaff37-50d4-4f86-98f0-839905a1bbd4 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:23:30.655617 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:23:30.701800 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0eeaff37-50d4-4f86-98f0-839905a1bbd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94926e51a8954fd8a836eba351a176de'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:23:30.709057 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0eeaff37-50d4-4f86-98f0-839905a1bbd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94926e51a8954fd8a836eba351a176de'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:23:30.715742 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0eeaff37-50d4-4f86-98f0-839905a1bbd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94926e51a8954fd8a836eba351a176de'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:23:30.716362 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0eeaff37-50d4-4f86-98f0-839905a1bbd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94926e51a8954fd8a836eba351a176de'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:23:30.716962 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0eeaff37-50d4-4f86-98f0-839905a1bbd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94926e51a8954fd8a836eba351a176de'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:23:30.734494 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1360/2747] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:23:30 2022] DELETE /identity/v3/users/94926e51a8954fd8a836eba351a176de => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:23:30.907755 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b61958-2e2c-4bd2-9495-9cb3f5750500 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:23:30.909242 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f2b61958-2e2c-4bd2-9495-9cb3f5750500 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:23:30.922570 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b61958-2e2c-4bd2-9495-9cb3f5750500 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:23:30.936040 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:23:30.982899 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c77e2-5b72-4347-bd66-788f3fe4d145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:23:30.983761 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c77e2-5b72-4347-bd66-788f3fe4d145 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:23:30.984047 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c77e2-5b72-4347-bd66-788f3fe4d145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:23:30.985191 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74c77e2-5b72-4347-bd66-788f3fe4d145 admin admin] PATH_INFO: `/v3/projects/010f41373a55498ab8a43a15609778de` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:23:30.985489 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74c77e2-5b72-4347-bd66-788f3fe4d145 admin admin] RBAC: Authorizing `identity:delete_project(project_id=010f41373a55498ab8a43a15609778de)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:23:30.987407 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74c77e2-5b72-4347-bd66-788f3fe4d145 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:23:31.035700 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e74c77e2-5b72-4347-bd66-788f3fe4d145 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '010f41373a55498ab8a43a15609778de', 'request_id': 'req-e74c77e2-5b72-4347-bd66-788f3fe4d145'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:23:31.042733 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e74c77e2-5b72-4347-bd66-788f3fe4d145 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '010f41373a55498ab8a43a15609778de', 'request_id': 'req-e74c77e2-5b72-4347-bd66-788f3fe4d145'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:23:31.048508 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e74c77e2-5b72-4347-bd66-788f3fe4d145 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '010f41373a55498ab8a43a15609778de', 'request_id': 'req-e74c77e2-5b72-4347-bd66-788f3fe4d145'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:23:31.059487 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e74c77e2-5b72-4347-bd66-788f3fe4d145 admin admin] The token cache is being invalidate because project 010f41373a55498ab8a43a15609778de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:23:31.059774 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e74c77e2-5b72-4347-bd66-788f3fe4d145 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:23:31.060410 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e74c77e2-5b72-4347-bd66-788f3fe4d145 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:23:31.061942 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1388/2748] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:23:30 2022] DELETE /identity/v3/projects/010f41373a55498ab8a43a15609778de => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:23:40.731147 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=25,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:23:41.056447 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=44,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:24:03.338061 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9de30c-92d4-48ef-b969-365fad48b87d None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:03.338604 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9de30c-92d4-48ef-b969-365fad48b87d None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:03.338604 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9de30c-92d4-48ef-b969-365fad48b87d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:03.356802 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-fe9de30c-92d4-48ef-b969-365fad48b87d None None] MFA Rules not processed for user `b1302d3b8649407ca010f12aad7a6597`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:24:03.357999 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-fe9de30c-92d4-48ef-b969-365fad48b87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:03.414698 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1361/2749] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:24:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:24:13.407887 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=25 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:24:41.133762 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c77e2-5b72-4347-bd66-788f3fe4d145 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:41.135398 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e74c77e2-5b72-4347-bd66-788f3fe4d145 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:41.195648 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74c77e2-5b72-4347-bd66-788f3fe4d145 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:41.199423 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b272c1-bf31-4cda-8d59-0f048edb0736 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:41.200075 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b272c1-bf31-4cda-8d59-0f048edb0736 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:41.200361 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b272c1-bf31-4cda-8d59-0f048edb0736 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:41.200642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b272c1-bf31-4cda-8d59-0f048edb0736 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:41.202484 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d7b272c1-bf31-4cda-8d59-0f048edb0736 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:41.279266 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b272c1-bf31-4cda-8d59-0f048edb0736 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:41.280832 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b272c1-bf31-4cda-8d59-0f048edb0736 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:41.293017 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1389/2750] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:24:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:24:42.479185 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9de30c-92d4-48ef-b969-365fad48b87d None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:42.489339 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9de30c-92d4-48ef-b969-365fad48b87d None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:42.493100 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d6ccf2-db1f-4abd-a9a0-462c5c70b469 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:42.494044 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6ccf2-db1f-4abd-a9a0-462c5c70b469 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:42.494363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6ccf2-db1f-4abd-a9a0-462c5c70b469 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:42.494669 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6ccf2-db1f-4abd-a9a0-462c5c70b469 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:42.496602 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e4d6ccf2-db1f-4abd-a9a0-462c5c70b469 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:42.509964 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d6ccf2-db1f-4abd-a9a0-462c5c70b469 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:42.511434 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d6ccf2-db1f-4abd-a9a0-462c5c70b469 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:42.522455 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1362/2751] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:24:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:24:51.287363 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=54 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:24:52.517395 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=6 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:24:54.563185 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b272c1-bf31-4cda-8d59-0f048edb0736 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:24:54.572417 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b272c1-bf31-4cda-8d59-0f048edb0736 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:24:54.576266 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a74437d-4045-4b37-a8f1-938e4b28c407 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:24:54.577414 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74437d-4045-4b37-a8f1-938e4b28c407 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:24:54.577414 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74437d-4045-4b37-a8f1-938e4b28c407 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:24:54.577414 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a74437d-4045-4b37-a8f1-938e4b28c407 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:24:54.579240 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8a74437d-4045-4b37-a8f1-938e4b28c407 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:24:54.667690 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a74437d-4045-4b37-a8f1-938e4b28c407 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:24:54.669199 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a74437d-4045-4b37-a8f1-938e4b28c407 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:24:54.681802 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1390/2752] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:24:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:25:01.827405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d6ccf2-db1f-4abd-a9a0-462c5c70b469 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:25:01.829373 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-e4d6ccf2-db1f-4abd-a9a0-462c5c70b469 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:25:01.844125 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d6ccf2-db1f-4abd-a9a0-462c5c70b469 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:25:01.849497 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ad77f-4763-47da-ba45-84376e34c375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:25:01.850544 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ad77f-4763-47da-ba45-84376e34c375 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:25:01.851043 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ad77f-4763-47da-ba45-84376e34c375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:25:01.851480 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ad77f-4763-47da-ba45-84376e34c375 admin admin] PATH_INFO: `/v3/users/af74115f423c43fdb2544876be4e5622` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:25:01.853638 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9ad77f-4763-47da-ba45-84376e34c375 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af74115f423c43fdb2544876be4e5622)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:25:01.856228 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9ad77f-4763-47da-ba45-84376e34c375 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:25:01.913652 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9d9ad77f-4763-47da-ba45-84376e34c375 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af74115f423c43fdb2544876be4e5622'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:25:01.921618 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9d9ad77f-4763-47da-ba45-84376e34c375 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af74115f423c43fdb2544876be4e5622'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:25:01.929009 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9d9ad77f-4763-47da-ba45-84376e34c375 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af74115f423c43fdb2544876be4e5622'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:25:01.929678 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9d9ad77f-4763-47da-ba45-84376e34c375 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af74115f423c43fdb2544876be4e5622'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:25:01.930351 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-9d9ad77f-4763-47da-ba45-84376e34c375 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af74115f423c43fdb2544876be4e5622'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:25:01.948642 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1363/2753] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:25:01 2022] DELETE /identity/v3/users/af74115f423c43fdb2544876be4e5622 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:25:02.138298 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a74437d-4045-4b37-a8f1-938e4b28c407 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:25:02.150124 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a74437d-4045-4b37-a8f1-938e4b28c407 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:25:02.243201 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152d3415-5447-4e18-aa4e-d0e589094652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:25:02.244083 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d3415-5447-4e18-aa4e-d0e589094652 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:25:02.244368 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d3415-5447-4e18-aa4e-d0e589094652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:25:02.244647 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d3415-5447-4e18-aa4e-d0e589094652 admin admin] PATH_INFO: `/v3/projects/f88bb4c8b75c4ad6be62f2cf5cf509c4` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:25:02.245854 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152d3415-5447-4e18-aa4e-d0e589094652 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f88bb4c8b75c4ad6be62f2cf5cf509c4)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:25:02.247835 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152d3415-5447-4e18-aa4e-d0e589094652 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:25:02.297545 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-152d3415-5447-4e18-aa4e-d0e589094652 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f88bb4c8b75c4ad6be62f2cf5cf509c4', 'request_id': 'req-152d3415-5447-4e18-aa4e-d0e589094652'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:25:02.305571 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-152d3415-5447-4e18-aa4e-d0e589094652 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f88bb4c8b75c4ad6be62f2cf5cf509c4', 'request_id': 'req-152d3415-5447-4e18-aa4e-d0e589094652'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:25:02.311789 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-152d3415-5447-4e18-aa4e-d0e589094652 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f88bb4c8b75c4ad6be62f2cf5cf509c4', 'request_id': 'req-152d3415-5447-4e18-aa4e-d0e589094652'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:25:02.323877 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-152d3415-5447-4e18-aa4e-d0e589094652 admin admin] The token cache is being invalidate because project f88bb4c8b75c4ad6be62f2cf5cf509c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:25:02.324172 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-152d3415-5447-4e18-aa4e-d0e589094652 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:25:02.324907 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-152d3415-5447-4e18-aa4e-d0e589094652 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:25:02.326480 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1391/2754] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:25:02 2022] DELETE /identity/v3/projects/f88bb4c8b75c4ad6be62f2cf5cf509c4 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:25:02.330525 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ad77f-4763-47da-ba45-84376e34c375 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:25:02.332047 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9d9ad77f-4763-47da-ba45-84376e34c375 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:25:02.346711 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ad77f-4763-47da-ba45-84376e34c375 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:25:02.407301 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e5a3e4-0777-4fdd-a34c-61b56f89f4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:25:02.408178 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e5a3e4-0777-4fdd-a34c-61b56f89f4f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:25:02.408464 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e5a3e4-0777-4fdd-a34c-61b56f89f4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:25:02.408743 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e5a3e4-0777-4fdd-a34c-61b56f89f4f9 admin admin] PATH_INFO: `/v3/users/ae06d907c76945e4ac189fed2b529850` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:25:02.409957 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e5a3e4-0777-4fdd-a34c-61b56f89f4f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae06d907c76945e4ac189fed2b529850)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:25:02.412038 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e5a3e4-0777-4fdd-a34c-61b56f89f4f9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:25:02.473169 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-20e5a3e4-0777-4fdd-a34c-61b56f89f4f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae06d907c76945e4ac189fed2b529850'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:25:02.481800 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-20e5a3e4-0777-4fdd-a34c-61b56f89f4f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae06d907c76945e4ac189fed2b529850'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:25:02.490631 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-20e5a3e4-0777-4fdd-a34c-61b56f89f4f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae06d907c76945e4ac189fed2b529850'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:25:02.491616 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-20e5a3e4-0777-4fdd-a34c-61b56f89f4f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae06d907c76945e4ac189fed2b529850'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:25:02.492227 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-20e5a3e4-0777-4fdd-a34c-61b56f89f4f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae06d907c76945e4ac189fed2b529850'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:25:02.502574 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:25:02.516052 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1364/2755] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:25:02 2022] DELETE /identity/v3/users/ae06d907c76945e4ac189fed2b529850 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:25:02.699979 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152d3415-5447-4e18-aa4e-d0e589094652 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:25:02.701498 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-152d3415-5447-4e18-aa4e-d0e589094652 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:25:02.717404 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152d3415-5447-4e18-aa4e-d0e589094652 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:25:02.739585 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:25:02.777702 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b19b6-4d96-4678-a93f-11362d4644eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:25:02.778559 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b19b6-4d96-4678-a93f-11362d4644eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:25:02.778838 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b19b6-4d96-4678-a93f-11362d4644eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:25:02.779139 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b19b6-4d96-4678-a93f-11362d4644eb admin admin] PATH_INFO: `/v3/projects/f849da19c4384675a073acbf977b2d4a` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:25:02.780360 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26b19b6-4d96-4678-a93f-11362d4644eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=f849da19c4384675a073acbf977b2d4a)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:25:02.782243 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26b19b6-4d96-4678-a93f-11362d4644eb admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:25:02.831160 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d26b19b6-4d96-4678-a93f-11362d4644eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f849da19c4384675a073acbf977b2d4a', 'request_id': 'req-d26b19b6-4d96-4678-a93f-11362d4644eb'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:25:02.839184 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d26b19b6-4d96-4678-a93f-11362d4644eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f849da19c4384675a073acbf977b2d4a', 'request_id': 'req-d26b19b6-4d96-4678-a93f-11362d4644eb'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:25:02.846455 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d26b19b6-4d96-4678-a93f-11362d4644eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f849da19c4384675a073acbf977b2d4a', 'request_id': 'req-d26b19b6-4d96-4678-a93f-11362d4644eb'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:25:02.858300 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d26b19b6-4d96-4678-a93f-11362d4644eb admin admin] The token cache is being invalidate because project f849da19c4384675a073acbf977b2d4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:25:02.858584 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d26b19b6-4d96-4678-a93f-11362d4644eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:25:02.859274 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d26b19b6-4d96-4678-a93f-11362d4644eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:25:02.860839 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1392/2756] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:25:02 2022] DELETE /identity/v3/projects/f849da19c4384675a073acbf977b2d4a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:25:05.069528 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba3021-6221-402b-ad75-b308a8065e4e None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:25:05.070023 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba3021-6221-402b-ad75-b308a8065e4e None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:25:05.070135 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ba3021-6221-402b-ad75-b308a8065e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:25:05.088840 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-20ba3021-6221-402b-ad75-b308a8065e4e None None] MFA Rules not processed for user `b1302d3b8649407ca010f12aad7a6597`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:25:05.090044 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-20ba3021-6221-402b-ad75-b308a8065e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:25:05.144800 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1365/2757] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:25:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:25:05.444461 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b19b6-4d96-4678-a93f-11362d4644eb admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:25:05.445848 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d26b19b6-4d96-4678-a93f-11362d4644eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:25:05.502075 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b19b6-4d96-4678-a93f-11362d4644eb admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:25:05.505675 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fda58a-dd9f-4a6d-889a-dffc2064a27d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:25:05.506318 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fda58a-dd9f-4a6d-889a-dffc2064a27d service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:25:05.506600 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fda58a-dd9f-4a6d-889a-dffc2064a27d service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:25:05.506898 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fda58a-dd9f-4a6d-889a-dffc2064a27d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:25:05.508751 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d7fda58a-dd9f-4a6d-889a-dffc2064a27d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:25:05.517947 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fda58a-dd9f-4a6d-889a-dffc2064a27d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:25:05.519406 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fda58a-dd9f-4a6d-889a-dffc2064a27d service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:25:05.527464 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1393/2758] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:25:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:25:05.852779 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba3021-6221-402b-ad75-b308a8065e4e None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:25:05.854160 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-20ba3021-6221-402b-ad75-b308a8065e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:25:05.923719 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ba3021-6221-402b-ad75-b308a8065e4e None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:25:05.928615 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d950b-2266-4577-a431-0d463e8c5812 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8293dab3ffd647f5927d323572fdbdce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:25:05.929260 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d950b-2266-4577-a431-0d463e8c5812 service placement] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:25:05.929538 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d950b-2266-4577-a431-0d463e8c5812 service placement] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:25:05.929812 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000d950b-2266-4577-a431-0d463e8c5812 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:25:05.931750 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-000d950b-2266-4577-a431-0d463e8c5812 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:25:05.942898 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000d950b-2266-4577-a431-0d463e8c5812 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:25:05.944466 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000d950b-2266-4577-a431-0d463e8c5812 service placement] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:25:05.955304 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1366/2759] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:25:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:25:07.937414 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fda58a-dd9f-4a6d-889a-dffc2064a27d service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:25:07.939097 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d7fda58a-dd9f-4a6d-889a-dffc2064a27d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:25:07.950169 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fda58a-dd9f-4a6d-889a-dffc2064a27d service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:25:07.954193 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c41ecc-7d7c-44f3-98b6-885f679403d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:25:07.954863 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c41ecc-7d7c-44f3-98b6-885f679403d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:25:07.955162 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c41ecc-7d7c-44f3-98b6-885f679403d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:25:07.955443 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c41ecc-7d7c-44f3-98b6-885f679403d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:25:07.957266 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d5c41ecc-7d7c-44f3-98b6-885f679403d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:25:07.969179 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c41ecc-7d7c-44f3-98b6-885f679403d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:25:07.971390 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c41ecc-7d7c-44f3-98b6-885f679403d2 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:25:07.981455 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1394/2760] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:25:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:25:15.950283 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=60,keystone:DELETE=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:25:17.976672 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=72,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:25:18.532404 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d950b-2266-4577-a431-0d463e8c5812 service placement] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:25:18.533847 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-000d950b-2266-4577-a431-0d463e8c5812 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:25:18.608861 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000d950b-2266-4577-a431-0d463e8c5812 service placement] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:25:18.614320 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7de2de4-c34a-4644-b29a-584ae4512ed5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '451a9232f95e494683d81b6078a318d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:25:18.615013 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de2de4-c34a-4644-b29a-584ae4512ed5 service nova] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:25:18.615295 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de2de4-c34a-4644-b29a-584ae4512ed5 service nova] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:25:18.615570 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7de2de4-c34a-4644-b29a-584ae4512ed5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:25:18.617404 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-c7de2de4-c34a-4644-b29a-584ae4512ed5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:25:18.632527 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7de2de4-c34a-4644-b29a-584ae4512ed5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:25:18.634790 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7de2de4-c34a-4644-b29a-584ae4512ed5 service nova] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:25:18.645995 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1367/2761] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:25:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:25:28.641286 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=34 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:26:07.919190 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c41ecc-7d7c-44f3-98b6-885f679403d2 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:26:07.920970 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d5c41ecc-7d7c-44f3-98b6-885f679403d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:26:07.940032 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c41ecc-7d7c-44f3-98b6-885f679403d2 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:26:07.946513 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2413aef2-974b-4f15-94ef-f6d0a818aa88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '451a9232f95e494683d81b6078a318d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:26:07.947399 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413aef2-974b-4f15-94ef-f6d0a818aa88 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:26:07.947734 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413aef2-974b-4f15-94ef-f6d0a818aa88 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:26:07.948092 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413aef2-974b-4f15-94ef-f6d0a818aa88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:26:07.950444 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2413aef2-974b-4f15-94ef-f6d0a818aa88 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:26:07.964167 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2413aef2-974b-4f15-94ef-f6d0a818aa88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:26:07.966459 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2413aef2-974b-4f15-94ef-f6d0a818aa88 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:26:07.982620 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1395/2762] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:26:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:26:10.671324 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457caca2-564a-4e66-876c-f53cbbe83961 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:26:10.671842 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457caca2-564a-4e66-876c-f53cbbe83961 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:26:10.671961 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457caca2-564a-4e66-876c-f53cbbe83961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:26:10.689719 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-457caca2-564a-4e66-876c-f53cbbe83961 None None] MFA Rules not processed for user `b1302d3b8649407ca010f12aad7a6597`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:26:10.690935 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-457caca2-564a-4e66-876c-f53cbbe83961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:26:10.697713 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1368/2763] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:26:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:26:17.973477 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:26:18.980121 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2413aef2-974b-4f15-94ef-f6d0a818aa88 service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:26:18.992808 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2413aef2-974b-4f15-94ef-f6d0a818aa88 service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:26:18.998301 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a12b24-b325-4ff3-b47b-f52d7edf4153 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '451a9232f95e494683d81b6078a318d0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'ResellerAdmin', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:26:18.998971 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a12b24-b325-4ff3-b47b-f52d7edf4153 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:26:18.999272 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a12b24-b325-4ff3-b47b-f52d7edf4153 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:26:18.999532 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a12b24-b325-4ff3-b47b-f52d7edf4153 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:26:19.001712 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b4a12b24-b325-4ff3-b47b-f52d7edf4153 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:26:19.014556 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a12b24-b325-4ff3-b47b-f52d7edf4153 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:26:19.016990 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a12b24-b325-4ff3-b47b-f52d7edf4153 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:26:19.027581 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1396/2764] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:26:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:26:20.684403 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:26:29.023112 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:27:19.453878 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457caca2-564a-4e66-876c-f53cbbe83961 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:27:19.522650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457caca2-564a-4e66-876c-f53cbbe83961 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:27:19.523258 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a12b24-b325-4ff3-b47b-f52d7edf4153 service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:27:19.525115 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b4a12b24-b325-4ff3-b47b-f52d7edf4153 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:27:19.529042 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff5d68-7bad-495c-8534-1cd7f0772a30 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8293dab3ffd647f5927d323572fdbdce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:27:19.530575 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff5d68-7bad-495c-8534-1cd7f0772a30 service placement] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:19.531101 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff5d68-7bad-495c-8534-1cd7f0772a30 service placement] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:19.531375 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff5d68-7bad-495c-8534-1cd7f0772a30 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:19.533385 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-23ff5d68-7bad-495c-8534-1cd7f0772a30 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:27:19.572820 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff5d68-7bad-495c-8534-1cd7f0772a30 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:27:19.574432 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff5d68-7bad-495c-8534-1cd7f0772a30 service placement] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:27:19.587745 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1369/2765] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:27:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:27:19.664891 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a12b24-b325-4ff3-b47b-f52d7edf4153 service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:27:19.669391 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167ad50f-d312-4dda-9992-2f6239c55f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:27:19.670303 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ad50f-d312-4dda-9992-2f6239c55f40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:19.670583 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ad50f-d312-4dda-9992-2f6239c55f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:19.670938 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ad50f-d312-4dda-9992-2f6239c55f40 admin admin] PATH_INFO: `/v3/users/b1302d3b8649407ca010f12aad7a6597` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:19.672072 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167ad50f-d312-4dda-9992-2f6239c55f40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1302d3b8649407ca010f12aad7a6597)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:27:19.673984 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167ad50f-d312-4dda-9992-2f6239c55f40 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:27:19.728372 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-167ad50f-d312-4dda-9992-2f6239c55f40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1302d3b8649407ca010f12aad7a6597'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:19.736637 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-167ad50f-d312-4dda-9992-2f6239c55f40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1302d3b8649407ca010f12aad7a6597'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:19.737350 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-167ad50f-d312-4dda-9992-2f6239c55f40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1302d3b8649407ca010f12aad7a6597'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:19.745024 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-167ad50f-d312-4dda-9992-2f6239c55f40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1302d3b8649407ca010f12aad7a6597'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:19.745928 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-167ad50f-d312-4dda-9992-2f6239c55f40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1302d3b8649407ca010f12aad7a6597'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:19.766413 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1397/2766] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:27:19 2022] DELETE /identity/v3/users/b1302d3b8649407ca010f12aad7a6597 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:27:19.952632 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff5d68-7bad-495c-8534-1cd7f0772a30 service placement] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:27:19.964312 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff5d68-7bad-495c-8534-1cd7f0772a30 service placement] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:27:20.023391 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa165bd-1a0c-4020-90ae-f4e17fd121be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:27:20.024255 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa165bd-1a0c-4020-90ae-f4e17fd121be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:20.024540 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa165bd-1a0c-4020-90ae-f4e17fd121be admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:20.024818 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa165bd-1a0c-4020-90ae-f4e17fd121be admin admin] PATH_INFO: `/v3/projects/f027c3b75bc6430abd765a26269f9058` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:20.026012 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa165bd-1a0c-4020-90ae-f4e17fd121be admin admin] RBAC: Authorizing `identity:delete_project(project_id=f027c3b75bc6430abd765a26269f9058)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:27:20.027967 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa165bd-1a0c-4020-90ae-f4e17fd121be admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:27:20.075677 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0fa165bd-1a0c-4020-90ae-f4e17fd121be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f027c3b75bc6430abd765a26269f9058', 'request_id': 'req-0fa165bd-1a0c-4020-90ae-f4e17fd121be'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:20.081664 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0fa165bd-1a0c-4020-90ae-f4e17fd121be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f027c3b75bc6430abd765a26269f9058', 'request_id': 'req-0fa165bd-1a0c-4020-90ae-f4e17fd121be'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:20.087990 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0fa165bd-1a0c-4020-90ae-f4e17fd121be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f027c3b75bc6430abd765a26269f9058', 'request_id': 'req-0fa165bd-1a0c-4020-90ae-f4e17fd121be'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:20.101532 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0fa165bd-1a0c-4020-90ae-f4e17fd121be admin admin] The token cache is being invalidate because project f027c3b75bc6430abd765a26269f9058 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:27:20.101830 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0fa165bd-1a0c-4020-90ae-f4e17fd121be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:20.102438 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-0fa165bd-1a0c-4020-90ae-f4e17fd121be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:20.104014 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1370/2767] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:27:19 2022] DELETE /identity/v3/projects/f027c3b75bc6430abd765a26269f9058 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:27:20.109288 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5184ba-e28b-48ef-95fc-c1df12250aaf None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:20.109582 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5184ba-e28b-48ef-95fc-c1df12250aaf None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:20.109867 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5184ba-e28b-48ef-95fc-c1df12250aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:20.131955 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:27:20.176291 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-cc5184ba-e28b-48ef-95fc-c1df12250aaf None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:27:20.177657 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cc5184ba-e28b-48ef-95fc-c1df12250aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:27:20.245855 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1398/2768] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:27:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:27:20.254406 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1211769-c0e5-401b-936b-b913ed7b1659 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:20.254694 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1211769-c0e5-401b-936b-b913ed7b1659 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:20.255009 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1211769-c0e5-401b-936b-b913ed7b1659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:20.274108 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-a1211769-c0e5-401b-936b-b913ed7b1659 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:27:20.275366 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a1211769-c0e5-401b-936b-b913ed7b1659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:27:20.283706 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1371/2769] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:27:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:27:20.288526 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5184ba-e28b-48ef-95fc-c1df12250aaf None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:27:20.298768 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5184ba-e28b-48ef-95fc-c1df12250aaf None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:27:20.303420 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eb5034-d8cd-48e8-8616-fc61b62add5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:27:20.304089 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb5034-d8cd-48e8-8616-fc61b62add5d admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:20.304377 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb5034-d8cd-48e8-8616-fc61b62add5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:20.304663 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb5034-d8cd-48e8-8616-fc61b62add5d admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:20.305386 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eb5034-d8cd-48e8-8616-fc61b62add5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:27:20.305698 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eb5034-d8cd-48e8-8616-fc61b62add5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:27:20.307347 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eb5034-d8cd-48e8-8616-fc61b62add5d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:27:20.321893 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1399/2770] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:27:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:27:20.326952 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1211769-c0e5-401b-936b-b913ed7b1659 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:27:20.336299 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1211769-c0e5-401b-936b-b913ed7b1659 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:27:20.341049 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a137b0-f1c9-4f6b-8e28-d85d3ded4128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:27:20.341701 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a137b0-f1c9-4f6b-8e28-d85d3ded4128 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:20.342063 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a137b0-f1c9-4f6b-8e28-d85d3ded4128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:20.342344 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a137b0-f1c9-4f6b-8e28-d85d3ded4128 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:20.343107 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a137b0-f1c9-4f6b-8e28-d85d3ded4128 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:27:20.345017 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a137b0-f1c9-4f6b-8e28-d85d3ded4128 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:27:20.365161 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:27:20.376230 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1372/2771] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:27:20 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:27:20.377372 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eb5034-d8cd-48e8-8616-fc61b62add5d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:27:20.387584 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eb5034-d8cd-48e8-8616-fc61b62add5d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:27:20.444590 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d9a590-9610-49d5-9d5f-b1674413348e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:27:20.445263 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d9a590-9610-49d5-9d5f-b1674413348e admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:20.445551 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d9a590-9610-49d5-9d5f-b1674413348e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:20.445829 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d9a590-9610-49d5-9d5f-b1674413348e admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:20.446514 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d9a590-9610-49d5-9d5f-b1674413348e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:27:20.448462 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d9a590-9610-49d5-9d5f-b1674413348e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:27:20.470090 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1400/2772] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:27:20 2022] POST /identity/v3/users => generated 283 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:27:20.474581 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a137b0-f1c9-4f6b-8e28-d85d3ded4128 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:27:20.485486 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a137b0-f1c9-4f6b-8e28-d85d3ded4128 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:27:20.490145 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c2cd90-dea3-471a-8835-de93f832cc79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:27:20.490812 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2cd90-dea3-471a-8835-de93f832cc79 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:20.491134 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2cd90-dea3-471a-8835-de93f832cc79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:20.491416 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2cd90-dea3-471a-8835-de93f832cc79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:20.492147 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c2cd90-dea3-471a-8835-de93f832cc79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:27:20.493756 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c2cd90-dea3-471a-8835-de93f832cc79 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:27:20.504953 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1373/2773] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:27:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:27:20.508991 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d9a590-9610-49d5-9d5f-b1674413348e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:27:20.521255 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d9a590-9610-49d5-9d5f-b1674413348e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:27:20.526316 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f9a7d-accf-4f52-b799-29157ba83243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:27:20.527116 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f9a7d-accf-4f52-b799-29157ba83243 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:20.527471 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f9a7d-accf-4f52-b799-29157ba83243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:20.527821 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f9a7d-accf-4f52-b799-29157ba83243 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:20.528509 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81f9a7d-accf-4f52-b799-29157ba83243 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:27:20.530149 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81f9a7d-accf-4f52-b799-29157ba83243 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:27:20.542273 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1401/2774] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:27:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:27:20.546468 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c2cd90-dea3-471a-8835-de93f832cc79 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:27:20.560658 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c2cd90-dea3-471a-8835-de93f832cc79 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:27:20.567101 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267cdd74-7d2d-4ffd-8c4f-cd418629de05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:27:20.567769 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267cdd74-7d2d-4ffd-8c4f-cd418629de05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:20.568571 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267cdd74-7d2d-4ffd-8c4f-cd418629de05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:20.568571 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267cdd74-7d2d-4ffd-8c4f-cd418629de05 admin admin] PATH_INFO: `/v3/projects/72f4fbaa42484639b89547fdec623b4e/users/478a7d42bdf3458599f908c10b660e72/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:20.584603 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267cdd74-7d2d-4ffd-8c4f-cd418629de05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72f4fbaa42484639b89547fdec623b4e, user_id=478a7d42bdf3458599f908c10b660e72, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:27:20.587277 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267cdd74-7d2d-4ffd-8c4f-cd418629de05 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:27:20.603217 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1374/2775] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:27:20 2022] PUT /identity/v3/projects/72f4fbaa42484639b89547fdec623b4e/users/478a7d42bdf3458599f908c10b660e72/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:27:20.614738 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b75b37-df29-477a-97d3-862bc584fb30 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:20.615079 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b75b37-df29-477a-97d3-862bc584fb30 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:20.615409 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b75b37-df29-477a-97d3-862bc584fb30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:20.645813 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-75b75b37-df29-477a-97d3-862bc584fb30 None None] MFA Rules not processed for user `478a7d42bdf3458599f908c10b660e72`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:27:20.647059 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-75b75b37-df29-477a-97d3-862bc584fb30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:27:20.693540 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:27:20.716571 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1402/2776] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:27:20 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:27:20.721624 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267cdd74-7d2d-4ffd-8c4f-cd418629de05 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:27:20.788293 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267cdd74-7d2d-4ffd-8c4f-cd418629de05 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:27:20.793045 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40666f31-52dc-42f2-a5cf-7e057468a4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:27:20.793703 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40666f31-52dc-42f2-a5cf-7e057468a4b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:20.794001 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40666f31-52dc-42f2-a5cf-7e057468a4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:20.794266 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40666f31-52dc-42f2-a5cf-7e057468a4b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:20.795518 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40666f31-52dc-42f2-a5cf-7e057468a4b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:27:20.797478 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40666f31-52dc-42f2-a5cf-7e057468a4b7 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:27:20.826235 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1375/2777] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:27:20 2022] POST /identity/v3/projects => generated 365 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:27:20.827991 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b75b37-df29-477a-97d3-862bc584fb30 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:27:20.838507 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b75b37-df29-477a-97d3-862bc584fb30 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:27:20.895463 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d021098-a8bb-4d6c-96a0-c6133dd57e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:27:20.896160 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d021098-a8bb-4d6c-96a0-c6133dd57e8c admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:20.896446 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d021098-a8bb-4d6c-96a0-c6133dd57e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:20.896721 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d021098-a8bb-4d6c-96a0-c6133dd57e8c admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:20.897416 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d021098-a8bb-4d6c-96a0-c6133dd57e8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:27:20.899375 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d021098-a8bb-4d6c-96a0-c6133dd57e8c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:27:20.919890 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1403/2778] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:27:20 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 27 15:27:20.924716 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40666f31-52dc-42f2-a5cf-7e057468a4b7 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:27:20.935082 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40666f31-52dc-42f2-a5cf-7e057468a4b7 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:27:20.939947 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9101b4d0-6318-4b88-b958-5d94fd015b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:27:20.940622 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101b4d0-6318-4b88-b958-5d94fd015b9d admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:20.940898 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101b4d0-6318-4b88-b958-5d94fd015b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:20.941179 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101b4d0-6318-4b88-b958-5d94fd015b9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:20.941898 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9101b4d0-6318-4b88-b958-5d94fd015b9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:27:20.943548 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9101b4d0-6318-4b88-b958-5d94fd015b9d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:27:20.954505 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1376/2779] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:27:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:27:20.958904 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d021098-a8bb-4d6c-96a0-c6133dd57e8c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:27:20.969748 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d021098-a8bb-4d6c-96a0-c6133dd57e8c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:27:20.974734 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29be8cb7-9a01-45b3-b15b-3c1f6b552686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:27:20.975642 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be8cb7-9a01-45b3-b15b-3c1f6b552686 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:20.975923 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be8cb7-9a01-45b3-b15b-3c1f6b552686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:20.976198 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29be8cb7-9a01-45b3-b15b-3c1f6b552686 admin admin] PATH_INFO: `/v3/projects/16e743fe353a45b18f7b8cf981d58b4b/users/9bb4e9c6de374acf895dcf188ebb3c11/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:20.992003 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29be8cb7-9a01-45b3-b15b-3c1f6b552686 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16e743fe353a45b18f7b8cf981d58b4b, user_id=9bb4e9c6de374acf895dcf188ebb3c11, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:27:20.994655 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29be8cb7-9a01-45b3-b15b-3c1f6b552686 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:27:21.009002 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1404/2780] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:27:20 2022] PUT /identity/v3/projects/16e743fe353a45b18f7b8cf981d58b4b/users/9bb4e9c6de374acf895dcf188ebb3c11/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:27:21.013342 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9101b4d0-6318-4b88-b958-5d94fd015b9d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:27:21.023668 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9101b4d0-6318-4b88-b958-5d94fd015b9d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:27:21.079850 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189ab2cf-3593-461c-b252-46bfe06b279b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:27:21.080433 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ab2cf-3593-461c-b252-46bfe06b279b admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:21.080718 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ab2cf-3593-461c-b252-46bfe06b279b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:21.080995 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ab2cf-3593-461c-b252-46bfe06b279b admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:21.081715 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189ab2cf-3593-461c-b252-46bfe06b279b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:27:21.083360 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189ab2cf-3593-461c-b252-46bfe06b279b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:27:21.094163 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1377/2781] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:27:21.098827 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29be8cb7-9a01-45b3-b15b-3c1f6b552686 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:27:21.110151 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29be8cb7-9a01-45b3-b15b-3c1f6b552686 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:27:21.114779 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd459be-1870-42df-acb6-33872e79c77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:27:21.115732 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd459be-1870-42df-acb6-33872e79c77b admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:21.115999 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd459be-1870-42df-acb6-33872e79c77b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:21.116258 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd459be-1870-42df-acb6-33872e79c77b admin admin] PATH_INFO: `/v3/projects/16e743fe353a45b18f7b8cf981d58b4b/users/9bb4e9c6de374acf895dcf188ebb3c11/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:21.118237 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd459be-1870-42df-acb6-33872e79c77b admin admin] RBAC: Authorizing `identity:create_grant(project_id=16e743fe353a45b18f7b8cf981d58b4b, user_id=9bb4e9c6de374acf895dcf188ebb3c11, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:27:21.120912 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd459be-1870-42df-acb6-33872e79c77b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:27:21.135499 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1405/2782] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:27:21 2022] PUT /identity/v3/projects/16e743fe353a45b18f7b8cf981d58b4b/users/9bb4e9c6de374acf895dcf188ebb3c11/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:27:21.145983 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeaff13-c97f-42d3-90af-ce967a7084d0 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:21.146282 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeaff13-c97f-42d3-90af-ce967a7084d0 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:21.146567 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeaff13-c97f-42d3-90af-ce967a7084d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:21.176165 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-eeeaff13-c97f-42d3-90af-ce967a7084d0 None None] MFA Rules not processed for user `9bb4e9c6de374acf895dcf188ebb3c11`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:27:21.177588 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-eeeaff13-c97f-42d3-90af-ce967a7084d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:27:21.201919 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:27:21.248827 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1378/2783] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:27:21 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:27:21.260468 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd459be-1870-42df-acb6-33872e79c77b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:27:21.262346 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0fd459be-1870-42df-acb6-33872e79c77b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:27:21.346564 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd459be-1870-42df-acb6-33872e79c77b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:27:21.350352 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08b0f68-7571-45b7-961d-5dd35f1cfed3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:27:21.351032 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b0f68-7571-45b7-961d-5dd35f1cfed3 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:21.351324 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b0f68-7571-45b7-961d-5dd35f1cfed3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:21.351607 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b0f68-7571-45b7-961d-5dd35f1cfed3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:21.362474 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08b0f68-7571-45b7-961d-5dd35f1cfed3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:27:21.363960 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08b0f68-7571-45b7-961d-5dd35f1cfed3 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:27:21.374404 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1406/2784] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:27:21 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:27:21.450653 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeaff13-c97f-42d3-90af-ce967a7084d0 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:27:21.461266 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeaff13-c97f-42d3-90af-ce967a7084d0 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:27:21.466523 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7db13f-e3f4-4e89-b42a-ae4df7a116e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:27:21.467377 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7db13f-e3f4-4e89-b42a-ae4df7a116e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:21.467662 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7db13f-e3f4-4e89-b42a-ae4df7a116e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:21.467939 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7db13f-e3f4-4e89-b42a-ae4df7a116e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:21.531381 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7db13f-e3f4-4e89-b42a-ae4df7a116e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:27:21.533004 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7db13f-e3f4-4e89-b42a-ae4df7a116e8 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:27:21.542777 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1379/2785] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:27:21 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:27:31.369243 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=87,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:27:31.538268 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=46 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:27:31.865823 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08b0f68-7571-45b7-961d-5dd35f1cfed3 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:27:31.929090 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08b0f68-7571-45b7-961d-5dd35f1cfed3 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:27:31.933570 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e2f74f-f1a3-4a2e-8146-cc3310880bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:27:31.934430 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2f74f-f1a3-4a2e-8146-cc3310880bd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:31.934710 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2f74f-f1a3-4a2e-8146-cc3310880bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:31.935011 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2f74f-f1a3-4a2e-8146-cc3310880bd4 admin admin] PATH_INFO: `/v3/users/478a7d42bdf3458599f908c10b660e72` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:31.936183 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e2f74f-f1a3-4a2e-8146-cc3310880bd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=478a7d42bdf3458599f908c10b660e72)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:27:31.938737 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e2f74f-f1a3-4a2e-8146-cc3310880bd4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:27:31.994595 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c0e2f74f-f1a3-4a2e-8146-cc3310880bd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '478a7d42bdf3458599f908c10b660e72'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:32.001981 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c0e2f74f-f1a3-4a2e-8146-cc3310880bd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '478a7d42bdf3458599f908c10b660e72'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:32.002674 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c0e2f74f-f1a3-4a2e-8146-cc3310880bd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '478a7d42bdf3458599f908c10b660e72'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:32.011336 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c0e2f74f-f1a3-4a2e-8146-cc3310880bd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '478a7d42bdf3458599f908c10b660e72'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:32.011915 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c0e2f74f-f1a3-4a2e-8146-cc3310880bd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '478a7d42bdf3458599f908c10b660e72'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:32.029247 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1407/2786] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:27:31 2022] DELETE /identity/v3/users/478a7d42bdf3458599f908c10b660e72 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:27:32.033875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060b5d1-7ab9-412a-8eeb-2f0d8b2ce4cf None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:32.034165 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060b5d1-7ab9-412a-8eeb-2f0d8b2ce4cf None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:32.034463 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9060b5d1-7ab9-412a-8eeb-2f0d8b2ce4cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:32.052278 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-9060b5d1-7ab9-412a-8eeb-2f0d8b2ce4cf None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:27:32.053460 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9060b5d1-7ab9-412a-8eeb-2f0d8b2ce4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:27:32.111217 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1380/2787] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:27:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:27:32.120763 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e2f74f-f1a3-4a2e-8146-cc3310880bd4 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:27:32.122127 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c0e2f74f-f1a3-4a2e-8146-cc3310880bd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:27:32.201370 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e2f74f-f1a3-4a2e-8146-cc3310880bd4 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:27:32.205085 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331eac7d-2d2e-4f12-9906-ba7d2da21747 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:27:32.205740 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331eac7d-2d2e-4f12-9906-ba7d2da21747 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:32.206023 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331eac7d-2d2e-4f12-9906-ba7d2da21747 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:32.206300 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331eac7d-2d2e-4f12-9906-ba7d2da21747 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:32.215595 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331eac7d-2d2e-4f12-9906-ba7d2da21747 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:27:32.217074 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331eac7d-2d2e-4f12-9906-ba7d2da21747 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:27:32.227302 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1408/2788] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:27:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:27:32.517157 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9060b5d1-7ab9-412a-8eeb-2f0d8b2ce4cf None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:27:32.518630 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9060b5d1-7ab9-412a-8eeb-2f0d8b2ce4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:27:32.533436 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9060b5d1-7ab9-412a-8eeb-2f0d8b2ce4cf None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:27:32.538950 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a8f51-395b-4d3f-8e0e-239cb22db2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:27:32.540623 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a8f51-395b-4d3f-8e0e-239cb22db2b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:32.540913 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a8f51-395b-4d3f-8e0e-239cb22db2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:32.541276 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8a8f51-395b-4d3f-8e0e-239cb22db2b0 admin admin] PATH_INFO: `/v3/projects/72f4fbaa42484639b89547fdec623b4e` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:32.542545 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8a8f51-395b-4d3f-8e0e-239cb22db2b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72f4fbaa42484639b89547fdec623b4e)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:27:32.544698 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8a8f51-395b-4d3f-8e0e-239cb22db2b0 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:27:32.593550 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4e8a8f51-395b-4d3f-8e0e-239cb22db2b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72f4fbaa42484639b89547fdec623b4e', 'request_id': 'req-4e8a8f51-395b-4d3f-8e0e-239cb22db2b0'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:32.599794 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4e8a8f51-395b-4d3f-8e0e-239cb22db2b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72f4fbaa42484639b89547fdec623b4e', 'request_id': 'req-4e8a8f51-395b-4d3f-8e0e-239cb22db2b0'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:32.606144 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4e8a8f51-395b-4d3f-8e0e-239cb22db2b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72f4fbaa42484639b89547fdec623b4e', 'request_id': 'req-4e8a8f51-395b-4d3f-8e0e-239cb22db2b0'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:32.620683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4e8a8f51-395b-4d3f-8e0e-239cb22db2b0 admin admin] The token cache is being invalidate because project 72f4fbaa42484639b89547fdec623b4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:27:32.620984 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4e8a8f51-395b-4d3f-8e0e-239cb22db2b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:32.621676 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-4e8a8f51-395b-4d3f-8e0e-239cb22db2b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:32.623265 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1381/2789] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:27:32 2022] DELETE /identity/v3/projects/72f4fbaa42484639b89547fdec623b4e => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:27:32.627367 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331eac7d-2d2e-4f12-9906-ba7d2da21747 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:27:32.632204 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:27:32.639047 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331eac7d-2d2e-4f12-9906-ba7d2da21747 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:27:32.696894 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e077b7-95b2-4e00-a127-da350dfb90f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:27:32.697774 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e077b7-95b2-4e00-a127-da350dfb90f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:32.698057 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e077b7-95b2-4e00-a127-da350dfb90f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:32.698337 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e077b7-95b2-4e00-a127-da350dfb90f6 admin admin] PATH_INFO: `/v3/users/9bb4e9c6de374acf895dcf188ebb3c11` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:32.699542 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e077b7-95b2-4e00-a127-da350dfb90f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bb4e9c6de374acf895dcf188ebb3c11)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:27:32.701458 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e077b7-95b2-4e00-a127-da350dfb90f6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:27:32.748248 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-12e077b7-95b2-4e00-a127-da350dfb90f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bb4e9c6de374acf895dcf188ebb3c11'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:32.754995 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-12e077b7-95b2-4e00-a127-da350dfb90f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bb4e9c6de374acf895dcf188ebb3c11'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:32.755664 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-12e077b7-95b2-4e00-a127-da350dfb90f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bb4e9c6de374acf895dcf188ebb3c11'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:32.763318 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-12e077b7-95b2-4e00-a127-da350dfb90f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bb4e9c6de374acf895dcf188ebb3c11'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:32.763978 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-12e077b7-95b2-4e00-a127-da350dfb90f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bb4e9c6de374acf895dcf188ebb3c11'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:32.782293 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1409/2790] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:27:32 2022] DELETE /identity/v3/users/9bb4e9c6de374acf895dcf188ebb3c11 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:27:33.055413 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a8f51-395b-4d3f-8e0e-239cb22db2b0 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:27:33.056831 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4e8a8f51-395b-4d3f-8e0e-239cb22db2b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:27:33.070197 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8a8f51-395b-4d3f-8e0e-239cb22db2b0 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:27:33.126639 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d7df25-1a21-4887-a292-13f410a0b65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:27:33.127520 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d7df25-1a21-4887-a292-13f410a0b65c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:33.127802 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d7df25-1a21-4887-a292-13f410a0b65c admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:33.128074 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d7df25-1a21-4887-a292-13f410a0b65c admin admin] PATH_INFO: `/v3/projects/16e743fe353a45b18f7b8cf981d58b4b` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:33.129334 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d7df25-1a21-4887-a292-13f410a0b65c admin admin] RBAC: Authorizing `identity:delete_project(project_id=16e743fe353a45b18f7b8cf981d58b4b)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:27:33.131177 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d7df25-1a21-4887-a292-13f410a0b65c admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:27:33.161977 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:27:33.179137 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b5d7df25-1a21-4887-a292-13f410a0b65c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16e743fe353a45b18f7b8cf981d58b4b', 'request_id': 'req-b5d7df25-1a21-4887-a292-13f410a0b65c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:33.184650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b5d7df25-1a21-4887-a292-13f410a0b65c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16e743fe353a45b18f7b8cf981d58b4b', 'request_id': 'req-b5d7df25-1a21-4887-a292-13f410a0b65c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:33.191621 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b5d7df25-1a21-4887-a292-13f410a0b65c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16e743fe353a45b18f7b8cf981d58b4b', 'request_id': 'req-b5d7df25-1a21-4887-a292-13f410a0b65c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:33.205015 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b5d7df25-1a21-4887-a292-13f410a0b65c admin admin] The token cache is being invalidate because project 16e743fe353a45b18f7b8cf981d58b4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:27:33.205301 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b5d7df25-1a21-4887-a292-13f410a0b65c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:33.205924 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b5d7df25-1a21-4887-a292-13f410a0b65c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:27:33.207488 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1382/2791] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:27:33 2022] DELETE /identity/v3/projects/16e743fe353a45b18f7b8cf981d58b4b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:27:33.212449 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb2fc9-924c-4901-823f-f242828afc20 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:33.212746 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb2fc9-924c-4901-823f-f242828afc20 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:33.213024 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb2fc9-924c-4901-823f-f242828afc20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:33.231563 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-46bb2fc9-924c-4901-823f-f242828afc20 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:27:33.232764 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-46bb2fc9-924c-4901-823f-f242828afc20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:27:33.293434 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1410/2792] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:27:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:27:33.303316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9abd0f-c902-47bb-805a-cea0ab8b0d5b None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:33.303606 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9abd0f-c902-47bb-805a-cea0ab8b0d5b None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:33.303885 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9abd0f-c902-47bb-805a-cea0ab8b0d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:33.322889 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-9d9abd0f-c902-47bb-805a-cea0ab8b0d5b None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:27:33.324083 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9d9abd0f-c902-47bb-805a-cea0ab8b0d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:27:33.331873 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1383/2793] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:27:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:27:33.336162 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bb2fc9-924c-4901-823f-f242828afc20 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:27:33.346349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bb2fc9-924c-4901-823f-f242828afc20 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:27:33.350860 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfed09b4-ae51-4ce2-898b-b27bfd0d9749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:27:33.351528 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfed09b4-ae51-4ce2-898b-b27bfd0d9749 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:33.351815 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfed09b4-ae51-4ce2-898b-b27bfd0d9749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:33.352090 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfed09b4-ae51-4ce2-898b-b27bfd0d9749 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:33.352838 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfed09b4-ae51-4ce2-898b-b27bfd0d9749 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:27:33.353142 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfed09b4-ae51-4ce2-898b-b27bfd0d9749 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:27:33.354719 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfed09b4-ae51-4ce2-898b-b27bfd0d9749 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:27:33.367204 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1411/2794] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:27:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:27:33.371568 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9abd0f-c902-47bb-805a-cea0ab8b0d5b None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:27:33.381393 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9abd0f-c902-47bb-805a-cea0ab8b0d5b None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:27:33.385860 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b2a83c-9e90-441f-a1c1-3efdc2544c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:27:33.386509 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b2a83c-9e90-441f-a1c1-3efdc2544c8b admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:33.386789 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b2a83c-9e90-441f-a1c1-3efdc2544c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:33.387092 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b2a83c-9e90-441f-a1c1-3efdc2544c8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:33.387787 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b2a83c-9e90-441f-a1c1-3efdc2544c8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:27:33.389684 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b2a83c-9e90-441f-a1c1-3efdc2544c8b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:27:33.415883 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1384/2795] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:27:33 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:27:33.420293 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfed09b4-ae51-4ce2-898b-b27bfd0d9749 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:27:33.431163 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfed09b4-ae51-4ce2-898b-b27bfd0d9749 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:27:33.436289 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:27:33.497460 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f35ff9f-f2d3-46e8-a36f-48371e69f96c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:27:33.498150 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f35ff9f-f2d3-46e8-a36f-48371e69f96c admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:33.498435 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f35ff9f-f2d3-46e8-a36f-48371e69f96c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:33.498749 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f35ff9f-f2d3-46e8-a36f-48371e69f96c admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:33.500104 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f35ff9f-f2d3-46e8-a36f-48371e69f96c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:27:33.502247 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f35ff9f-f2d3-46e8-a36f-48371e69f96c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:27:33.523673 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1412/2796] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:27:33 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:27:33.528138 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b2a83c-9e90-441f-a1c1-3efdc2544c8b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:27:33.539778 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b2a83c-9e90-441f-a1c1-3efdc2544c8b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:27:33.544995 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd1260-329b-40ed-bea0-6741e9cdce91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:27:33.546232 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd1260-329b-40ed-bea0-6741e9cdce91 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:33.546526 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd1260-329b-40ed-bea0-6741e9cdce91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:33.546811 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd1260-329b-40ed-bea0-6741e9cdce91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:33.547669 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfd1260-329b-40ed-bea0-6741e9cdce91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:27:33.549515 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfd1260-329b-40ed-bea0-6741e9cdce91 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:27:33.562166 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1385/2797] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:27:33.566724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f35ff9f-f2d3-46e8-a36f-48371e69f96c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:27:33.577449 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f35ff9f-f2d3-46e8-a36f-48371e69f96c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:27:33.582299 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf435cc-a352-45ea-8c63-e5447b7cbf59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:27:33.582975 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf435cc-a352-45ea-8c63-e5447b7cbf59 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:33.583263 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf435cc-a352-45ea-8c63-e5447b7cbf59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:33.583531 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf435cc-a352-45ea-8c63-e5447b7cbf59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:33.584243 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf435cc-a352-45ea-8c63-e5447b7cbf59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:27:33.585828 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf435cc-a352-45ea-8c63-e5447b7cbf59 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:27:33.597468 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1413/2798] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:27:33.601847 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd1260-329b-40ed-bea0-6741e9cdce91 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:27:33.612909 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd1260-329b-40ed-bea0-6741e9cdce91 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:27:33.617592 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e02a17-36a9-4a52-b8d0-d838b9132f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:27:33.618472 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e02a17-36a9-4a52-b8d0-d838b9132f53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:33.618752 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e02a17-36a9-4a52-b8d0-d838b9132f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:33.619057 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e02a17-36a9-4a52-b8d0-d838b9132f53 admin admin] PATH_INFO: `/v3/projects/f64f695034cb448a9426e2de8cac5942/users/0fc1372e68fc4ef2b814f2a1f91b2a36/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:33.633993 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e02a17-36a9-4a52-b8d0-d838b9132f53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f64f695034cb448a9426e2de8cac5942, user_id=0fc1372e68fc4ef2b814f2a1f91b2a36, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:27:33.636607 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e02a17-36a9-4a52-b8d0-d838b9132f53 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:27:33.649852 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1386/2799] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:27:33 2022] PUT /identity/v3/projects/f64f695034cb448a9426e2de8cac5942/users/0fc1372e68fc4ef2b814f2a1f91b2a36/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:27:33.661369 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd99f1-3edc-4309-9f64-986dc754464d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:33.661933 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd99f1-3edc-4309-9f64-986dc754464d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:33.661933 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd99f1-3edc-4309-9f64-986dc754464d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:33.689262 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-02dd99f1-3edc-4309-9f64-986dc754464d None None] MFA Rules not processed for user `0fc1372e68fc4ef2b814f2a1f91b2a36`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:27:33.690455 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-02dd99f1-3edc-4309-9f64-986dc754464d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:27:33.755627 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1414/2800] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:27:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:27:33.760261 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e02a17-36a9-4a52-b8d0-d838b9132f53 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:27:33.823743 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e02a17-36a9-4a52-b8d0-d838b9132f53 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:27:33.829105 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5cae66-a389-40cb-bbd9-f14626f19620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:27:33.829843 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5cae66-a389-40cb-bbd9-f14626f19620 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:33.830128 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5cae66-a389-40cb-bbd9-f14626f19620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:33.830406 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5cae66-a389-40cb-bbd9-f14626f19620 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:33.831201 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5cae66-a389-40cb-bbd9-f14626f19620 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:27:33.833271 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5cae66-a389-40cb-bbd9-f14626f19620 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:27:33.862150 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1387/2801] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:27:33 2022] POST /identity/v3/projects => generated 375 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:27:33.866280 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd99f1-3edc-4309-9f64-986dc754464d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:27:33.877763 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd99f1-3edc-4309-9f64-986dc754464d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:27:33.935316 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e501321-88ee-4750-ac17-a3e8430922fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:27:33.935898 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e501321-88ee-4750-ac17-a3e8430922fe admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:33.936182 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e501321-88ee-4750-ac17-a3e8430922fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:33.936493 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e501321-88ee-4750-ac17-a3e8430922fe admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:33.937599 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e501321-88ee-4750-ac17-a3e8430922fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:27:33.939536 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e501321-88ee-4750-ac17-a3e8430922fe admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:27:33.960761 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1415/2802] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:27:33 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:27:33.964764 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5cae66-a389-40cb-bbd9-f14626f19620 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:27:33.976099 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5cae66-a389-40cb-bbd9-f14626f19620 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:27:33.980791 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8374af64-d991-4097-b589-6c17612d3651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:27:33.981440 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8374af64-d991-4097-b589-6c17612d3651 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:33.981733 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8374af64-d991-4097-b589-6c17612d3651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:33.982012 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8374af64-d991-4097-b589-6c17612d3651 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:33.982737 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8374af64-d991-4097-b589-6c17612d3651 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:27:33.985080 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8374af64-d991-4097-b589-6c17612d3651 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:27:33.996037 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1388/2803] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:27:34.000341 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e501321-88ee-4750-ac17-a3e8430922fe admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:27:34.011007 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e501321-88ee-4750-ac17-a3e8430922fe admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:27:34.017131 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d520460-24d8-45f8-8fd0-43b29dc36fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:27:34.018026 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d520460-24d8-45f8-8fd0-43b29dc36fad admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:34.018313 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d520460-24d8-45f8-8fd0-43b29dc36fad admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:34.018594 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d520460-24d8-45f8-8fd0-43b29dc36fad admin admin] PATH_INFO: `/v3/projects/e1606f17b0ae4f0bb5d1fabe612106c4/users/b1e0c9f26bfe4bbfab7cb22c72d37632/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:34.021073 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:27:34.036748 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d520460-24d8-45f8-8fd0-43b29dc36fad admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1606f17b0ae4f0bb5d1fabe612106c4, user_id=b1e0c9f26bfe4bbfab7cb22c72d37632, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:27:34.039466 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d520460-24d8-45f8-8fd0-43b29dc36fad admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:27:34.053349 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1416/2804] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:27:33 2022] PUT /identity/v3/projects/e1606f17b0ae4f0bb5d1fabe612106c4/users/b1e0c9f26bfe4bbfab7cb22c72d37632/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:27:34.057571 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8374af64-d991-4097-b589-6c17612d3651 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:27:34.068535 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8374af64-d991-4097-b589-6c17612d3651 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:27:34.089016 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:27:34.129021 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec222f0d-855e-43aa-a191-76bb94713aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:27:34.129704 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec222f0d-855e-43aa-a191-76bb94713aa8 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:34.130010 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec222f0d-855e-43aa-a191-76bb94713aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:34.130272 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec222f0d-855e-43aa-a191-76bb94713aa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:34.131048 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec222f0d-855e-43aa-a191-76bb94713aa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:27:34.132692 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec222f0d-855e-43aa-a191-76bb94713aa8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:27:34.144606 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1389/2805] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:27:34 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:27:34.149135 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d520460-24d8-45f8-8fd0-43b29dc36fad admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:27:34.160550 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d520460-24d8-45f8-8fd0-43b29dc36fad admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:27:34.165393 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543be1d5-bb11-474d-b735-32a81fa46258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:27:34.166290 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543be1d5-bb11-474d-b735-32a81fa46258 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:34.166576 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543be1d5-bb11-474d-b735-32a81fa46258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:34.166873 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543be1d5-bb11-474d-b735-32a81fa46258 admin admin] PATH_INFO: `/v3/projects/e1606f17b0ae4f0bb5d1fabe612106c4/users/b1e0c9f26bfe4bbfab7cb22c72d37632/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:34.168906 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543be1d5-bb11-474d-b735-32a81fa46258 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1606f17b0ae4f0bb5d1fabe612106c4, user_id=b1e0c9f26bfe4bbfab7cb22c72d37632, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:27:34.171644 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543be1d5-bb11-474d-b735-32a81fa46258 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:27:34.182424 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1417/2806] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:27:34 2022] PUT /identity/v3/projects/e1606f17b0ae4f0bb5d1fabe612106c4/users/b1e0c9f26bfe4bbfab7cb22c72d37632/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:27:34.193493 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a50e2d3-ad24-48e2-85ad-f90f7356c743 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:34.193710 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a50e2d3-ad24-48e2-85ad-f90f7356c743 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:34.193989 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a50e2d3-ad24-48e2-85ad-f90f7356c743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:34.223106 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-1a50e2d3-ad24-48e2-85ad-f90f7356c743 None None] MFA Rules not processed for user `b1e0c9f26bfe4bbfab7cb22c72d37632`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:27:34.224310 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-1a50e2d3-ad24-48e2-85ad-f90f7356c743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:27:34.293899 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1390/2807] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:27:34 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:27:34.305402 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543be1d5-bb11-474d-b735-32a81fa46258 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:27:34.307161 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-543be1d5-bb11-474d-b735-32a81fa46258 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:27:34.369189 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543be1d5-bb11-474d-b735-32a81fa46258 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:27:34.372896 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8736ee26-4f47-4537-8e8a-0dbfff84d08a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:27:34.373561 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8736ee26-4f47-4537-8e8a-0dbfff84d08a service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:34.374367 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8736ee26-4f47-4537-8e8a-0dbfff84d08a service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:34.374668 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8736ee26-4f47-4537-8e8a-0dbfff84d08a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:34.434179 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8736ee26-4f47-4537-8e8a-0dbfff84d08a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:27:34.435641 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8736ee26-4f47-4537-8e8a-0dbfff84d08a service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:27:34.444975 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1418/2808] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:27:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:27:39.067190 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a50e2d3-ad24-48e2-85ad-f90f7356c743 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:27:39.078673 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a50e2d3-ad24-48e2-85ad-f90f7356c743 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:27:39.084086 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa9aa53-e576-4b18-aa06-10f6be497ed9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:27:39.084732 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa9aa53-e576-4b18-aa06-10f6be497ed9 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:27:39.085017 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa9aa53-e576-4b18-aa06-10f6be497ed9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:27:39.085295 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa9aa53-e576-4b18-aa06-10f6be497ed9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:27:39.095119 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa9aa53-e576-4b18-aa06-10f6be497ed9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:27:39.096804 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa9aa53-e576-4b18-aa06-10f6be497ed9 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:27:39.107001 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1391/2809] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:27:39 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:27:44.440973 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=60,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:27:49.102512 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=58 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:28:03.937907 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8736ee26-4f47-4537-8e8a-0dbfff84d08a service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:04.006781 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8736ee26-4f47-4537-8e8a-0dbfff84d08a service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:04.011291 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b504cff7-aae0-452c-b5f6-815d09366e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:04.012290 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b504cff7-aae0-452c-b5f6-815d09366e6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:04.012714 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b504cff7-aae0-452c-b5f6-815d09366e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:04.012851 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b504cff7-aae0-452c-b5f6-815d09366e6d admin admin] PATH_INFO: `/v3/users/0fc1372e68fc4ef2b814f2a1f91b2a36` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:04.014054 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b504cff7-aae0-452c-b5f6-815d09366e6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fc1372e68fc4ef2b814f2a1f91b2a36)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:04.016108 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b504cff7-aae0-452c-b5f6-815d09366e6d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:04.073478 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b504cff7-aae0-452c-b5f6-815d09366e6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fc1372e68fc4ef2b814f2a1f91b2a36'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:04.080378 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b504cff7-aae0-452c-b5f6-815d09366e6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fc1372e68fc4ef2b814f2a1f91b2a36'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:04.081061 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b504cff7-aae0-452c-b5f6-815d09366e6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fc1372e68fc4ef2b814f2a1f91b2a36'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:04.089987 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b504cff7-aae0-452c-b5f6-815d09366e6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fc1372e68fc4ef2b814f2a1f91b2a36'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:04.090627 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b504cff7-aae0-452c-b5f6-815d09366e6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fc1372e68fc4ef2b814f2a1f91b2a36'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:04.109223 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1419/2810] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:28:03 2022] DELETE /identity/v3/users/0fc1372e68fc4ef2b814f2a1f91b2a36 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:28:04.113785 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c47f85-85e9-4005-a83f-04d758ccb442 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:04.114338 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c47f85-85e9-4005-a83f-04d758ccb442 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:04.114485 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c47f85-85e9-4005-a83f-04d758ccb442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:04.131588 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-63c47f85-85e9-4005-a83f-04d758ccb442 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:28:04.132773 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-63c47f85-85e9-4005-a83f-04d758ccb442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:28:04.197379 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1392/2811] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:28:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:28:04.207548 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b504cff7-aae0-452c-b5f6-815d09366e6d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:04.209672 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b504cff7-aae0-452c-b5f6-815d09366e6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:28:04.266400 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b504cff7-aae0-452c-b5f6-815d09366e6d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:04.269986 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb9786-5b04-4d05-be68-8d0e9c4d415c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:04.271223 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb9786-5b04-4d05-be68-8d0e9c4d415c service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:04.271505 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb9786-5b04-4d05-be68-8d0e9c4d415c service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:04.271779 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb9786-5b04-4d05-be68-8d0e9c4d415c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:04.280640 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfb9786-5b04-4d05-be68-8d0e9c4d415c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:04.282055 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfb9786-5b04-4d05-be68-8d0e9c4d415c service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:04.291889 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1420/2812] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:28:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:28:04.571953 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c47f85-85e9-4005-a83f-04d758ccb442 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:04.573526 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-63c47f85-85e9-4005-a83f-04d758ccb442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:28:04.587422 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c47f85-85e9-4005-a83f-04d758ccb442 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:04.592461 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5874676-c55a-408b-98c0-4b93925beb79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:04.593330 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5874676-c55a-408b-98c0-4b93925beb79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:04.593637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5874676-c55a-408b-98c0-4b93925beb79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:04.593913 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5874676-c55a-408b-98c0-4b93925beb79 admin admin] PATH_INFO: `/v3/projects/f64f695034cb448a9426e2de8cac5942` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:04.595379 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5874676-c55a-408b-98c0-4b93925beb79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f64f695034cb448a9426e2de8cac5942)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:04.597641 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5874676-c55a-408b-98c0-4b93925beb79 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:04.650814 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a5874676-c55a-408b-98c0-4b93925beb79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f64f695034cb448a9426e2de8cac5942', 'request_id': 'req-a5874676-c55a-408b-98c0-4b93925beb79'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:04.656121 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a5874676-c55a-408b-98c0-4b93925beb79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f64f695034cb448a9426e2de8cac5942', 'request_id': 'req-a5874676-c55a-408b-98c0-4b93925beb79'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:04.663158 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a5874676-c55a-408b-98c0-4b93925beb79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f64f695034cb448a9426e2de8cac5942', 'request_id': 'req-a5874676-c55a-408b-98c0-4b93925beb79'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:04.676868 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a5874676-c55a-408b-98c0-4b93925beb79 admin admin] The token cache is being invalidate because project f64f695034cb448a9426e2de8cac5942 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:28:04.677146 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a5874676-c55a-408b-98c0-4b93925beb79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:04.677650 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a5874676-c55a-408b-98c0-4b93925beb79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:04.679233 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1393/2813] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:28:04 2022] DELETE /identity/v3/projects/f64f695034cb448a9426e2de8cac5942 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:28:04.683260 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb9786-5b04-4d05-be68-8d0e9c4d415c service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:04.692987 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb9786-5b04-4d05-be68-8d0e9c4d415c service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:04.712494 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:28:04.756135 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8c1fa2-2a1c-4970-9901-8dbe050aa197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:04.756919 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c1fa2-2a1c-4970-9901-8dbe050aa197 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:04.757196 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c1fa2-2a1c-4970-9901-8dbe050aa197 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:04.757468 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c1fa2-2a1c-4970-9901-8dbe050aa197 admin admin] PATH_INFO: `/v3/users/b1e0c9f26bfe4bbfab7cb22c72d37632` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:04.758645 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8c1fa2-2a1c-4970-9901-8dbe050aa197 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1e0c9f26bfe4bbfab7cb22c72d37632)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:04.760571 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8c1fa2-2a1c-4970-9901-8dbe050aa197 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:04.806936 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1a8c1fa2-2a1c-4970-9901-8dbe050aa197 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1e0c9f26bfe4bbfab7cb22c72d37632'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:04.813625 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1a8c1fa2-2a1c-4970-9901-8dbe050aa197 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1e0c9f26bfe4bbfab7cb22c72d37632'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:04.814282 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1a8c1fa2-2a1c-4970-9901-8dbe050aa197 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1e0c9f26bfe4bbfab7cb22c72d37632'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:04.821278 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1a8c1fa2-2a1c-4970-9901-8dbe050aa197 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1e0c9f26bfe4bbfab7cb22c72d37632'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:04.821991 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1a8c1fa2-2a1c-4970-9901-8dbe050aa197 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1e0c9f26bfe4bbfab7cb22c72d37632'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:04.839515 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1421/2814] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:28:04 2022] DELETE /identity/v3/users/b1e0c9f26bfe4bbfab7cb22c72d37632 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:28:05.112484 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5874676-c55a-408b-98c0-4b93925beb79 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:05.114036 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a5874676-c55a-408b-98c0-4b93925beb79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:28:05.127885 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5874676-c55a-408b-98c0-4b93925beb79 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:05.187334 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd7f8dd-daaa-467a-a855-e93ec715bfef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:05.188182 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7f8dd-daaa-467a-a855-e93ec715bfef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:05.188461 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7f8dd-daaa-467a-a855-e93ec715bfef admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:05.188741 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7f8dd-daaa-467a-a855-e93ec715bfef admin admin] PATH_INFO: `/v3/projects/e1606f17b0ae4f0bb5d1fabe612106c4` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:05.189920 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd7f8dd-daaa-467a-a855-e93ec715bfef admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1606f17b0ae4f0bb5d1fabe612106c4)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:05.191860 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd7f8dd-daaa-467a-a855-e93ec715bfef admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:05.223078 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:28:05.242564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3cd7f8dd-daaa-467a-a855-e93ec715bfef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1606f17b0ae4f0bb5d1fabe612106c4', 'request_id': 'req-3cd7f8dd-daaa-467a-a855-e93ec715bfef'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:05.248258 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3cd7f8dd-daaa-467a-a855-e93ec715bfef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1606f17b0ae4f0bb5d1fabe612106c4', 'request_id': 'req-3cd7f8dd-daaa-467a-a855-e93ec715bfef'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:05.254582 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3cd7f8dd-daaa-467a-a855-e93ec715bfef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1606f17b0ae4f0bb5d1fabe612106c4', 'request_id': 'req-3cd7f8dd-daaa-467a-a855-e93ec715bfef'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:05.271184 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3cd7f8dd-daaa-467a-a855-e93ec715bfef admin admin] The token cache is being invalidate because project e1606f17b0ae4f0bb5d1fabe612106c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:28:05.271546 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3cd7f8dd-daaa-467a-a855-e93ec715bfef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:05.272337 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-3cd7f8dd-daaa-467a-a855-e93ec715bfef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:05.274397 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1394/2815] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:28:05 2022] DELETE /identity/v3/projects/e1606f17b0ae4f0bb5d1fabe612106c4 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:28:05.279216 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497dec4c-67b4-4a25-af5e-6848de858b42 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:05.280016 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497dec4c-67b4-4a25-af5e-6848de858b42 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:05.280016 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497dec4c-67b4-4a25-af5e-6848de858b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:05.298087 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-497dec4c-67b4-4a25-af5e-6848de858b42 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:28:05.299316 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-497dec4c-67b4-4a25-af5e-6848de858b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:28:05.360529 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1422/2816] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:28:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:28:05.369682 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d1ce0-d8b1-43be-8709-77a73fedff23 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:05.369975 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d1ce0-d8b1-43be-8709-77a73fedff23 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:05.370250 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d1ce0-d8b1-43be-8709-77a73fedff23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:05.388998 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-579d1ce0-d8b1-43be-8709-77a73fedff23 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:28:05.390185 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-579d1ce0-d8b1-43be-8709-77a73fedff23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:28:05.398553 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1395/2817] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:28:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:28:05.402941 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497dec4c-67b4-4a25-af5e-6848de858b42 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:05.412766 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497dec4c-67b4-4a25-af5e-6848de858b42 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:05.417359 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db60a0-cf84-443d-adb2-f11bbe922cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:05.418019 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db60a0-cf84-443d-adb2-f11bbe922cdb admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:05.420011 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db60a0-cf84-443d-adb2-f11bbe922cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:05.420011 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db60a0-cf84-443d-adb2-f11bbe922cdb admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:05.420011 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db60a0-cf84-443d-adb2-f11bbe922cdb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:28:05.420011 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db60a0-cf84-443d-adb2-f11bbe922cdb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:05.421181 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db60a0-cf84-443d-adb2-f11bbe922cdb admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:05.433617 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1423/2818] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:28:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:28:05.438133 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d1ce0-d8b1-43be-8709-77a73fedff23 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:05.448535 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d1ce0-d8b1-43be-8709-77a73fedff23 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:05.453215 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2462e004-6552-4f0e-b99d-36d68c3b2a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:05.453869 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462e004-6552-4f0e-b99d-36d68c3b2a23 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:05.454148 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462e004-6552-4f0e-b99d-36d68c3b2a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:05.454423 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462e004-6552-4f0e-b99d-36d68c3b2a23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:05.455171 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2462e004-6552-4f0e-b99d-36d68c3b2a23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:05.457073 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2462e004-6552-4f0e-b99d-36d68c3b2a23 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:05.484077 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1396/2819] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:28:05 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:28:05.488381 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db60a0-cf84-443d-adb2-f11bbe922cdb admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:05.498743 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db60a0-cf84-443d-adb2-f11bbe922cdb admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:05.524828 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:28:05.558297 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91b6b5-79d9-4788-8863-1281007bd26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:05.558985 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91b6b5-79d9-4788-8863-1281007bd26d admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:05.559317 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91b6b5-79d9-4788-8863-1281007bd26d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:05.559638 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df91b6b5-79d9-4788-8863-1281007bd26d admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:05.560393 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df91b6b5-79d9-4788-8863-1281007bd26d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:05.562538 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df91b6b5-79d9-4788-8863-1281007bd26d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:05.584127 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1424/2820] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:28:05 2022] POST /identity/v3/users => generated 283 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:28:05.588258 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2462e004-6552-4f0e-b99d-36d68c3b2a23 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:05.598729 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2462e004-6552-4f0e-b99d-36d68c3b2a23 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:05.603402 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da2e659-5d5e-4c3f-93c7-dd041e1d01d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:05.604048 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2e659-5d5e-4c3f-93c7-dd041e1d01d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:05.604331 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2e659-5d5e-4c3f-93c7-dd041e1d01d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:05.604604 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da2e659-5d5e-4c3f-93c7-dd041e1d01d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:05.605353 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da2e659-5d5e-4c3f-93c7-dd041e1d01d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:05.606970 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da2e659-5d5e-4c3f-93c7-dd041e1d01d9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:05.618380 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1397/2821] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:28:05.624637 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91b6b5-79d9-4788-8863-1281007bd26d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:05.638633 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df91b6b5-79d9-4788-8863-1281007bd26d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:05.643271 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7055ea-805c-43e7-a857-17003a8185d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:05.643937 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7055ea-805c-43e7-a857-17003a8185d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:05.644219 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7055ea-805c-43e7-a857-17003a8185d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:05.644493 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7055ea-805c-43e7-a857-17003a8185d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:05.645225 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7055ea-805c-43e7-a857-17003a8185d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:05.647092 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7055ea-805c-43e7-a857-17003a8185d8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:05.660738 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1425/2822] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:28:05.665227 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da2e659-5d5e-4c3f-93c7-dd041e1d01d9 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:05.675363 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da2e659-5d5e-4c3f-93c7-dd041e1d01d9 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:05.680728 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c8bdd4-36ec-4fa1-86c0-ab5064488d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:05.681598 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8bdd4-36ec-4fa1-86c0-ab5064488d0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:05.681879 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8bdd4-36ec-4fa1-86c0-ab5064488d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:05.682158 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8bdd4-36ec-4fa1-86c0-ab5064488d0e admin admin] PATH_INFO: `/v3/projects/a75204484e6145eaa934665358436c1b/users/51c490e1842c4c7aa36c517598ca5eb4/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:05.697175 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c8bdd4-36ec-4fa1-86c0-ab5064488d0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a75204484e6145eaa934665358436c1b, user_id=51c490e1842c4c7aa36c517598ca5eb4, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:05.699872 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c8bdd4-36ec-4fa1-86c0-ab5064488d0e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:05.712639 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1398/2823] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:28:05 2022] PUT /identity/v3/projects/a75204484e6145eaa934665358436c1b/users/51c490e1842c4c7aa36c517598ca5eb4/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:28:05.723342 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f0c2d-3aa6-4927-8cc0-c2edd164d27e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:05.724142 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f0c2d-3aa6-4927-8cc0-c2edd164d27e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:05.724428 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f0c2d-3aa6-4927-8cc0-c2edd164d27e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:05.750449 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f47f0c2d-3aa6-4927-8cc0-c2edd164d27e None None] MFA Rules not processed for user `51c490e1842c4c7aa36c517598ca5eb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:28:05.751683 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f47f0c2d-3aa6-4927-8cc0-c2edd164d27e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:28:05.817115 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1426/2824] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:28:05 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:28:05.821836 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c8bdd4-36ec-4fa1-86c0-ab5064488d0e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:05.888588 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c8bdd4-36ec-4fa1-86c0-ab5064488d0e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:05.894011 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d81464-9fdb-4e4b-9a47-7ee176116376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:05.894707 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d81464-9fdb-4e4b-9a47-7ee176116376 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:05.895028 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d81464-9fdb-4e4b-9a47-7ee176116376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:05.895308 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d81464-9fdb-4e4b-9a47-7ee176116376 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:05.895998 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d81464-9fdb-4e4b-9a47-7ee176116376 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:05.898030 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d81464-9fdb-4e4b-9a47-7ee176116376 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:05.926739 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1399/2825] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:28:05 2022] POST /identity/v3/projects => generated 365 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:28:05.931851 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47f0c2d-3aa6-4927-8cc0-c2edd164d27e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:05.943633 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47f0c2d-3aa6-4927-8cc0-c2edd164d27e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:06.016820 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c91282-d965-4b38-812b-fcb4add4143e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:06.024409 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c91282-d965-4b38-812b-fcb4add4143e admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:06.024697 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c91282-d965-4b38-812b-fcb4add4143e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:06.024974 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c91282-d965-4b38-812b-fcb4add4143e admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:06.025684 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c91282-d965-4b38-812b-fcb4add4143e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:06.027671 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c91282-d965-4b38-812b-fcb4add4143e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:06.056904 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1427/2826] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:28:05 2022] POST /identity/v3/users => generated 289 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:28:06.060920 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d81464-9fdb-4e4b-9a47-7ee176116376 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:06.072738 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d81464-9fdb-4e4b-9a47-7ee176116376 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:06.077639 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff408da-608d-4c5a-923b-518556511e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:06.078356 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff408da-608d-4c5a-923b-518556511e85 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:06.078719 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff408da-608d-4c5a-923b-518556511e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:06.079048 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff408da-608d-4c5a-923b-518556511e85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:06.079812 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff408da-608d-4c5a-923b-518556511e85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:06.081559 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff408da-608d-4c5a-923b-518556511e85 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:06.093891 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1400/2827] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:28:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:28:06.098679 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c91282-d965-4b38-812b-fcb4add4143e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:06.109375 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c91282-d965-4b38-812b-fcb4add4143e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:06.115029 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f20198-1041-4bf8-9365-e79cec68a460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:06.115911 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f20198-1041-4bf8-9365-e79cec68a460 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:06.116193 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f20198-1041-4bf8-9365-e79cec68a460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:06.116470 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f20198-1041-4bf8-9365-e79cec68a460 admin admin] PATH_INFO: `/v3/projects/13357556c1394f7083dbfcfdbe85b2dd/users/bb53ba603f804909a24b27e200ef8925/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:06.132066 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f20198-1041-4bf8-9365-e79cec68a460 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13357556c1394f7083dbfcfdbe85b2dd, user_id=bb53ba603f804909a24b27e200ef8925, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:06.134688 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f20198-1041-4bf8-9365-e79cec68a460 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:06.149773 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1428/2828] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:28:06 2022] PUT /identity/v3/projects/13357556c1394f7083dbfcfdbe85b2dd/users/bb53ba603f804909a24b27e200ef8925/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:28:06.154262 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff408da-608d-4c5a-923b-518556511e85 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:06.167637 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff408da-608d-4c5a-923b-518556511e85 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:06.187312 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:28:06.227163 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98ec03a-fdd1-4f33-a76c-be5b16bcec85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:06.227874 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ec03a-fdd1-4f33-a76c-be5b16bcec85 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:06.228199 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ec03a-fdd1-4f33-a76c-be5b16bcec85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:06.228491 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ec03a-fdd1-4f33-a76c-be5b16bcec85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:06.229261 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98ec03a-fdd1-4f33-a76c-be5b16bcec85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:06.231065 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98ec03a-fdd1-4f33-a76c-be5b16bcec85 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:06.241869 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1401/2829] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:28:06 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:28:06.245899 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f20198-1041-4bf8-9365-e79cec68a460 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:06.251352 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:28:06.257500 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f20198-1041-4bf8-9365-e79cec68a460 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:06.262547 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3e369f-2d39-41a9-91d2-b2edcf99b7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:06.263451 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e369f-2d39-41a9-91d2-b2edcf99b7e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:06.263728 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e369f-2d39-41a9-91d2-b2edcf99b7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:06.263998 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3e369f-2d39-41a9-91d2-b2edcf99b7e3 admin admin] PATH_INFO: `/v3/projects/13357556c1394f7083dbfcfdbe85b2dd/users/bb53ba603f804909a24b27e200ef8925/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:06.265932 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3e369f-2d39-41a9-91d2-b2edcf99b7e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13357556c1394f7083dbfcfdbe85b2dd, user_id=bb53ba603f804909a24b27e200ef8925, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:06.268591 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3e369f-2d39-41a9-91d2-b2edcf99b7e3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:06.279594 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1429/2830] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:28:06 2022] PUT /identity/v3/projects/13357556c1394f7083dbfcfdbe85b2dd/users/bb53ba603f804909a24b27e200ef8925/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:28:06.291356 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe381b-d347-476f-97bd-abadc2e44ea2 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:06.291649 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe381b-d347-476f-97bd-abadc2e44ea2 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:06.291931 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe381b-d347-476f-97bd-abadc2e44ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:06.319180 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-9cfe381b-d347-476f-97bd-abadc2e44ea2 None None] MFA Rules not processed for user `bb53ba603f804909a24b27e200ef8925`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:28:06.320416 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9cfe381b-d347-476f-97bd-abadc2e44ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:28:06.405638 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1402/2831] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:28:06 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:28:06.417771 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3e369f-2d39-41a9-91d2-b2edcf99b7e3 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:06.419519 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-db3e369f-2d39-41a9-91d2-b2edcf99b7e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:28:06.477692 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3e369f-2d39-41a9-91d2-b2edcf99b7e3 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:06.482013 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf7093-06e3-4688-86a5-31bf0c232ef7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:06.482670 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf7093-06e3-4688-86a5-31bf0c232ef7 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:06.482995 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf7093-06e3-4688-86a5-31bf0c232ef7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:06.483279 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf7093-06e3-4688-86a5-31bf0c232ef7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:06.494108 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaf7093-06e3-4688-86a5-31bf0c232ef7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:06.495656 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaf7093-06e3-4688-86a5-31bf0c232ef7 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:06.506266 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1430/2832] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:28:06 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:28:06.588299 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfe381b-d347-476f-97bd-abadc2e44ea2 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:06.599027 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfe381b-d347-476f-97bd-abadc2e44ea2 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:06.603131 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3620e5ff-6284-4e1f-9071-80d2ead168bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:06.603853 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3620e5ff-6284-4e1f-9071-80d2ead168bc service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:06.604140 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3620e5ff-6284-4e1f-9071-80d2ead168bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:06.604438 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3620e5ff-6284-4e1f-9071-80d2ead168bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:06.661804 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3620e5ff-6284-4e1f-9071-80d2ead168bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:06.663188 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3620e5ff-6284-4e1f-9071-80d2ead168bc service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:06.671860 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1403/2833] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:28:06 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:28:11.455722 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf7093-06e3-4688-86a5-31bf0c232ef7 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:11.532907 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf7093-06e3-4688-86a5-31bf0c232ef7 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:11.538121 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf647936-5e2b-4c25-bef0-2af953e94cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:11.539037 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf647936-5e2b-4c25-bef0-2af953e94cf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:11.539384 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf647936-5e2b-4c25-bef0-2af953e94cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:11.539705 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf647936-5e2b-4c25-bef0-2af953e94cf0 admin admin] PATH_INFO: `/v3/users/51c490e1842c4c7aa36c517598ca5eb4` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:11.541188 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf647936-5e2b-4c25-bef0-2af953e94cf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51c490e1842c4c7aa36c517598ca5eb4)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:11.543631 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf647936-5e2b-4c25-bef0-2af953e94cf0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:11.593887 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-cf647936-5e2b-4c25-bef0-2af953e94cf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51c490e1842c4c7aa36c517598ca5eb4'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:11.601728 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-cf647936-5e2b-4c25-bef0-2af953e94cf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51c490e1842c4c7aa36c517598ca5eb4'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:11.602446 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-cf647936-5e2b-4c25-bef0-2af953e94cf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51c490e1842c4c7aa36c517598ca5eb4'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:11.610734 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-cf647936-5e2b-4c25-bef0-2af953e94cf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51c490e1842c4c7aa36c517598ca5eb4'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:11.611458 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-cf647936-5e2b-4c25-bef0-2af953e94cf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51c490e1842c4c7aa36c517598ca5eb4'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:11.629067 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1431/2834] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:28:11 2022] DELETE /identity/v3/users/51c490e1842c4c7aa36c517598ca5eb4 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:28:11.633852 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8408703-a4a4-4d19-adc9-c73aa86874c1 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:11.634389 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8408703-a4a4-4d19-adc9-c73aa86874c1 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:11.634389 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8408703-a4a4-4d19-adc9-c73aa86874c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:11.652323 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-b8408703-a4a4-4d19-adc9-c73aa86874c1 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:28:11.653507 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b8408703-a4a4-4d19-adc9-c73aa86874c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:28:11.693617 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=100 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:28:11.712603 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1404/2835] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:28:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:28:11.722584 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf647936-5e2b-4c25-bef0-2af953e94cf0 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:11.724072 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cf647936-5e2b-4c25-bef0-2af953e94cf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:28:11.737561 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:28:11.783512 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf647936-5e2b-4c25-bef0-2af953e94cf0 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:11.787528 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1af8cde-fe9c-4641-8841-94e37c8966ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:11.788179 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af8cde-fe9c-4641-8841-94e37c8966ab service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:11.788465 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af8cde-fe9c-4641-8841-94e37c8966ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:11.788746 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af8cde-fe9c-4641-8841-94e37c8966ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:11.800246 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1af8cde-fe9c-4641-8841-94e37c8966ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:11.801944 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1af8cde-fe9c-4641-8841-94e37c8966ab service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:11.811785 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1432/2836] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:28:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:28:12.117692 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8408703-a4a4-4d19-adc9-c73aa86874c1 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:12.119614 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-b8408703-a4a4-4d19-adc9-c73aa86874c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:28:12.132752 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8408703-a4a4-4d19-adc9-c73aa86874c1 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:12.137854 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31096617-42fa-4f1b-b0ba-db40ae7a6eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:12.138713 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31096617-42fa-4f1b-b0ba-db40ae7a6eb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:12.139026 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31096617-42fa-4f1b-b0ba-db40ae7a6eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:12.139308 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31096617-42fa-4f1b-b0ba-db40ae7a6eb9 admin admin] PATH_INFO: `/v3/projects/a75204484e6145eaa934665358436c1b` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:12.140921 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31096617-42fa-4f1b-b0ba-db40ae7a6eb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a75204484e6145eaa934665358436c1b)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:12.142840 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31096617-42fa-4f1b-b0ba-db40ae7a6eb9 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:12.192091 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-31096617-42fa-4f1b-b0ba-db40ae7a6eb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a75204484e6145eaa934665358436c1b', 'request_id': 'req-31096617-42fa-4f1b-b0ba-db40ae7a6eb9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:12.198997 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-31096617-42fa-4f1b-b0ba-db40ae7a6eb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a75204484e6145eaa934665358436c1b', 'request_id': 'req-31096617-42fa-4f1b-b0ba-db40ae7a6eb9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:12.205277 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-31096617-42fa-4f1b-b0ba-db40ae7a6eb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a75204484e6145eaa934665358436c1b', 'request_id': 'req-31096617-42fa-4f1b-b0ba-db40ae7a6eb9'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:12.217689 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-31096617-42fa-4f1b-b0ba-db40ae7a6eb9 admin admin] The token cache is being invalidate because project a75204484e6145eaa934665358436c1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:28:12.217970 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-31096617-42fa-4f1b-b0ba-db40ae7a6eb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:12.218642 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-31096617-42fa-4f1b-b0ba-db40ae7a6eb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:12.220315 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1405/2837] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:28:12 2022] DELETE /identity/v3/projects/a75204484e6145eaa934665358436c1b => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:28:12.225554 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1af8cde-fe9c-4641-8841-94e37c8966ab service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:12.236010 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1af8cde-fe9c-4641-8841-94e37c8966ab service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:12.293572 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12dd5b5-bf42-4d31-bb7d-507e1db5d388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:12.294506 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12dd5b5-bf42-4d31-bb7d-507e1db5d388 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:12.294927 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12dd5b5-bf42-4d31-bb7d-507e1db5d388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:12.294927 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12dd5b5-bf42-4d31-bb7d-507e1db5d388 admin admin] PATH_INFO: `/v3/users/bb53ba603f804909a24b27e200ef8925` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:12.296423 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12dd5b5-bf42-4d31-bb7d-507e1db5d388 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb53ba603f804909a24b27e200ef8925)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:12.298106 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12dd5b5-bf42-4d31-bb7d-507e1db5d388 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:12.351267 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e12dd5b5-bf42-4d31-bb7d-507e1db5d388 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb53ba603f804909a24b27e200ef8925'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:12.358087 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e12dd5b5-bf42-4d31-bb7d-507e1db5d388 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb53ba603f804909a24b27e200ef8925'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:12.358782 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e12dd5b5-bf42-4d31-bb7d-507e1db5d388 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb53ba603f804909a24b27e200ef8925'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:12.367369 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e12dd5b5-bf42-4d31-bb7d-507e1db5d388 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb53ba603f804909a24b27e200ef8925'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:12.368016 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e12dd5b5-bf42-4d31-bb7d-507e1db5d388 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb53ba603f804909a24b27e200ef8925'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:12.388698 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1433/2838] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:28:12 2022] DELETE /identity/v3/users/bb53ba603f804909a24b27e200ef8925 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:28:12.673610 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31096617-42fa-4f1b-b0ba-db40ae7a6eb9 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:12.675103 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-31096617-42fa-4f1b-b0ba-db40ae7a6eb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:28:12.691197 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31096617-42fa-4f1b-b0ba-db40ae7a6eb9 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:12.753355 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00d72ac-3a2a-4be7-a3d6-871c084f9746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:12.754112 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d72ac-3a2a-4be7-a3d6-871c084f9746 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:12.754400 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d72ac-3a2a-4be7-a3d6-871c084f9746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:12.755198 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d72ac-3a2a-4be7-a3d6-871c084f9746 admin admin] PATH_INFO: `/v3/projects/13357556c1394f7083dbfcfdbe85b2dd` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:12.756588 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00d72ac-3a2a-4be7-a3d6-871c084f9746 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13357556c1394f7083dbfcfdbe85b2dd)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:12.758509 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00d72ac-3a2a-4be7-a3d6-871c084f9746 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:12.805635 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b00d72ac-3a2a-4be7-a3d6-871c084f9746 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13357556c1394f7083dbfcfdbe85b2dd', 'request_id': 'req-b00d72ac-3a2a-4be7-a3d6-871c084f9746'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:12.811101 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b00d72ac-3a2a-4be7-a3d6-871c084f9746 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13357556c1394f7083dbfcfdbe85b2dd', 'request_id': 'req-b00d72ac-3a2a-4be7-a3d6-871c084f9746'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:12.817376 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b00d72ac-3a2a-4be7-a3d6-871c084f9746 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13357556c1394f7083dbfcfdbe85b2dd', 'request_id': 'req-b00d72ac-3a2a-4be7-a3d6-871c084f9746'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:12.825720 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:28:12.830871 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b00d72ac-3a2a-4be7-a3d6-871c084f9746 admin admin] The token cache is being invalidate because project 13357556c1394f7083dbfcfdbe85b2dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80394) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:28:12.831173 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b00d72ac-3a2a-4be7-a3d6-871c084f9746 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:12.832093 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-b00d72ac-3a2a-4be7-a3d6-871c084f9746 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:12.834945 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1406/2839] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:28:12 2022] DELETE /identity/v3/projects/13357556c1394f7083dbfcfdbe85b2dd => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:28:12.839646 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba62ea5-f26d-44cc-9623-e9581ab7a10e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:12.840113 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba62ea5-f26d-44cc-9623-e9581ab7a10e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:12.840230 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba62ea5-f26d-44cc-9623-e9581ab7a10e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:12.858996 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9ba62ea5-f26d-44cc-9623-e9581ab7a10e None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:28:12.860187 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9ba62ea5-f26d-44cc-9623-e9581ab7a10e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:28:12.889133 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:28:12.921355 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1434/2840] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:28:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:28:12.929109 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef007a3-c1f0-406f-b252-87e392feeafb None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:12.929398 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef007a3-c1f0-406f-b252-87e392feeafb None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:12.929673 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef007a3-c1f0-406f-b252-87e392feeafb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:12.948113 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-4ef007a3-c1f0-406f-b252-87e392feeafb None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:28:12.949311 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-4ef007a3-c1f0-406f-b252-87e392feeafb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:28:12.957143 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1407/2841] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:28:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:28:12.961872 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba62ea5-f26d-44cc-9623-e9581ab7a10e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:12.971893 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba62ea5-f26d-44cc-9623-e9581ab7a10e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:12.977346 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9561b35c-08b5-4afb-a55d-0276f6b8dcfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:12.978012 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561b35c-08b5-4afb-a55d-0276f6b8dcfc admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:12.978301 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561b35c-08b5-4afb-a55d-0276f6b8dcfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:12.978663 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561b35c-08b5-4afb-a55d-0276f6b8dcfc admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:12.979528 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9561b35c-08b5-4afb-a55d-0276f6b8dcfc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:28:12.979845 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9561b35c-08b5-4afb-a55d-0276f6b8dcfc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:12.981447 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9561b35c-08b5-4afb-a55d-0276f6b8dcfc admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:12.993558 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1435/2842] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:28:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:28:12.998202 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef007a3-c1f0-406f-b252-87e392feeafb None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:13.008712 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef007a3-c1f0-406f-b252-87e392feeafb None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:13.013728 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee3469-d941-41cb-bb10-8d6e8e1bdf59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:13.014278 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee3469-d941-41cb-bb10-8d6e8e1bdf59 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:13.014624 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee3469-d941-41cb-bb10-8d6e8e1bdf59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:13.014989 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee3469-d941-41cb-bb10-8d6e8e1bdf59 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:13.015754 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee3469-d941-41cb-bb10-8d6e8e1bdf59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:13.017894 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee3469-d941-41cb-bb10-8d6e8e1bdf59 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:13.048825 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1408/2843] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:28:12 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:28:13.055757 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9561b35c-08b5-4afb-a55d-0276f6b8dcfc admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:13.065017 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9561b35c-08b5-4afb-a55d-0276f6b8dcfc admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:13.123432 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e4a64-24dd-4a4b-9e53-ef1cf49bc1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:13.124098 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e4a64-24dd-4a4b-9e53-ef1cf49bc1ce admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:13.124375 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e4a64-24dd-4a4b-9e53-ef1cf49bc1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:13.124646 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e4a64-24dd-4a4b-9e53-ef1cf49bc1ce admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:13.125344 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010e4a64-24dd-4a4b-9e53-ef1cf49bc1ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:13.127263 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010e4a64-24dd-4a4b-9e53-ef1cf49bc1ce admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:13.147771 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1436/2844] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:28:13 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:28:13.152040 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee3469-d941-41cb-bb10-8d6e8e1bdf59 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:13.163704 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee3469-d941-41cb-bb10-8d6e8e1bdf59 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:13.168364 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f03010-03d2-4371-af36-c5c3fa0ef6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:13.169030 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f03010-03d2-4371-af36-c5c3fa0ef6ea admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:13.169310 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f03010-03d2-4371-af36-c5c3fa0ef6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:13.169593 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f03010-03d2-4371-af36-c5c3fa0ef6ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:13.170310 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f03010-03d2-4371-af36-c5c3fa0ef6ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:13.171926 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f03010-03d2-4371-af36-c5c3fa0ef6ea admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:13.185369 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1409/2845] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:28:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:28:13.189587 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e4a64-24dd-4a4b-9e53-ef1cf49bc1ce admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:13.200503 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e4a64-24dd-4a4b-9e53-ef1cf49bc1ce admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:13.205300 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57faf027-b55b-4e4a-ac2e-5263666af791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:13.205964 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57faf027-b55b-4e4a-ac2e-5263666af791 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:13.206276 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57faf027-b55b-4e4a-ac2e-5263666af791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:13.206534 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57faf027-b55b-4e4a-ac2e-5263666af791 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:13.207364 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57faf027-b55b-4e4a-ac2e-5263666af791 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:13.209565 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57faf027-b55b-4e4a-ac2e-5263666af791 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:13.219962 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1437/2846] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:28:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:28:13.225020 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f03010-03d2-4371-af36-c5c3fa0ef6ea admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:13.236003 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f03010-03d2-4371-af36-c5c3fa0ef6ea admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:13.240913 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502135fa-930c-4c16-8c76-679264e848a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:13.241801 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502135fa-930c-4c16-8c76-679264e848a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:13.242089 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502135fa-930c-4c16-8c76-679264e848a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:13.242375 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502135fa-930c-4c16-8c76-679264e848a5 admin admin] PATH_INFO: `/v3/projects/cc216780212c4dfa880d7ccc439f416f/users/de1f7ef8655744cf9244d3988a9fc04c/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:13.257640 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502135fa-930c-4c16-8c76-679264e848a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc216780212c4dfa880d7ccc439f416f, user_id=de1f7ef8655744cf9244d3988a9fc04c, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:13.260411 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502135fa-930c-4c16-8c76-679264e848a5 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:13.273891 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1410/2847] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:28:13 2022] PUT /identity/v3/projects/cc216780212c4dfa880d7ccc439f416f/users/de1f7ef8655744cf9244d3988a9fc04c/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:28:13.284990 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7311f113-0e13-45e1-817e-f40293578194 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:13.285375 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7311f113-0e13-45e1-817e-f40293578194 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:13.285846 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7311f113-0e13-45e1-817e-f40293578194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:13.314116 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7311f113-0e13-45e1-817e-f40293578194 None None] MFA Rules not processed for user `de1f7ef8655744cf9244d3988a9fc04c`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:28:13.315362 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7311f113-0e13-45e1-817e-f40293578194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:28:13.384280 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1438/2848] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:28:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:28:13.388951 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502135fa-930c-4c16-8c76-679264e848a5 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:13.462655 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502135fa-930c-4c16-8c76-679264e848a5 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:13.467708 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a0d8e9-4284-4a7c-ad66-0b0fe194ae91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:13.468519 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0d8e9-4284-4a7c-ad66-0b0fe194ae91 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:13.468923 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0d8e9-4284-4a7c-ad66-0b0fe194ae91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:13.469245 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a0d8e9-4284-4a7c-ad66-0b0fe194ae91 admin admin] PATH_INFO: `/v3/projects` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:13.470146 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a0d8e9-4284-4a7c-ad66-0b0fe194ae91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:13.472378 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a0d8e9-4284-4a7c-ad66-0b0fe194ae91 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:13.500319 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1411/2849] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:28:13 2022] POST /identity/v3/projects => generated 381 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:28:13.504362 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7311f113-0e13-45e1-817e-f40293578194 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:13.516507 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7311f113-0e13-45e1-817e-f40293578194 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:13.544462 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:28:13.587801 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff748ee9-e31e-415c-bced-3612221fe28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:13.588690 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff748ee9-e31e-415c-bced-3612221fe28b admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:13.589000 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff748ee9-e31e-415c-bced-3612221fe28b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:13.589280 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff748ee9-e31e-415c-bced-3612221fe28b admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:13.589978 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff748ee9-e31e-415c-bced-3612221fe28b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:13.592033 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff748ee9-e31e-415c-bced-3612221fe28b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:13.614891 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1439/2850] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:28:13 2022] POST /identity/v3/users => generated 297 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:28:13.619253 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a0d8e9-4284-4a7c-ad66-0b0fe194ae91 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:13.630705 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a0d8e9-4284-4a7c-ad66-0b0fe194ae91 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:13.635517 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5569b52a-9bb2-46db-a8cf-292ff7ed974d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:13.636178 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5569b52a-9bb2-46db-a8cf-292ff7ed974d admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:13.636458 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5569b52a-9bb2-46db-a8cf-292ff7ed974d admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:13.636732 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5569b52a-9bb2-46db-a8cf-292ff7ed974d admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:13.637446 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5569b52a-9bb2-46db-a8cf-292ff7ed974d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:13.639082 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5569b52a-9bb2-46db-a8cf-292ff7ed974d admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:13.649879 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1412/2851] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:28:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:28:13.654253 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff748ee9-e31e-415c-bced-3612221fe28b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:13.664801 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff748ee9-e31e-415c-bced-3612221fe28b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:13.669917 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5407064-3eee-4e39-b012-695c244dcc34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:13.670814 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5407064-3eee-4e39-b012-695c244dcc34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:13.671122 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5407064-3eee-4e39-b012-695c244dcc34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:13.671403 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5407064-3eee-4e39-b012-695c244dcc34 admin admin] PATH_INFO: `/v3/projects/b96259df88e44bd6ae04e3f54202618d/users/a75ee2bc9d2c4e6a82fc46217970945d/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:13.688690 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5407064-3eee-4e39-b012-695c244dcc34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b96259df88e44bd6ae04e3f54202618d, user_id=a75ee2bc9d2c4e6a82fc46217970945d, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:13.691802 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5407064-3eee-4e39-b012-695c244dcc34 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:13.709027 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1440/2852] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:28:13 2022] PUT /identity/v3/projects/b96259df88e44bd6ae04e3f54202618d/users/a75ee2bc9d2c4e6a82fc46217970945d/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:28:13.713559 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5569b52a-9bb2-46db-a8cf-292ff7ed974d admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:13.725194 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5569b52a-9bb2-46db-a8cf-292ff7ed974d admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:13.787283 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29101f1-ab07-4773-a9ac-a17c4f2bf026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:13.788160 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29101f1-ab07-4773-a9ac-a17c4f2bf026 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:13.788587 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29101f1-ab07-4773-a9ac-a17c4f2bf026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:13.788885 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29101f1-ab07-4773-a9ac-a17c4f2bf026 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:13.789621 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29101f1-ab07-4773-a9ac-a17c4f2bf026 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:13.791261 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29101f1-ab07-4773-a9ac-a17c4f2bf026 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:13.802671 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1413/2853] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:28:13 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:28:13.807021 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5407064-3eee-4e39-b012-695c244dcc34 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:13.821673 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5407064-3eee-4e39-b012-695c244dcc34 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:13.826467 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef05f9c2-6efe-4014-bcea-92bf42338c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:13.827395 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05f9c2-6efe-4014-bcea-92bf42338c4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:13.827682 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05f9c2-6efe-4014-bcea-92bf42338c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:13.827959 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05f9c2-6efe-4014-bcea-92bf42338c4d admin admin] PATH_INFO: `/v3/projects/b96259df88e44bd6ae04e3f54202618d/users/a75ee2bc9d2c4e6a82fc46217970945d/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:13.829920 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef05f9c2-6efe-4014-bcea-92bf42338c4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b96259df88e44bd6ae04e3f54202618d, user_id=a75ee2bc9d2c4e6a82fc46217970945d, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:13.833500 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef05f9c2-6efe-4014-bcea-92bf42338c4d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:13.845186 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1441/2854] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:28:13 2022] PUT /identity/v3/projects/b96259df88e44bd6ae04e3f54202618d/users/a75ee2bc9d2c4e6a82fc46217970945d/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 27 15:28:13.908720 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec6e91-7a20-475a-aee0-1f32cd1cf968 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:13.909018 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec6e91-7a20-475a-aee0-1f32cd1cf968 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:13.909343 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ec6e91-7a20-475a-aee0-1f32cd1cf968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:13.916796 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:28:13.940081 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-07ec6e91-7a20-475a-aee0-1f32cd1cf968 None None] MFA Rules not processed for user `a75ee2bc9d2c4e6a82fc46217970945d`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:28:13.941339 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-07ec6e91-7a20-475a-aee0-1f32cd1cf968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:28:14.012308 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1414/2855] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:28:13 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:28:14.023931 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef05f9c2-6efe-4014-bcea-92bf42338c4d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:14.025666 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ef05f9c2-6efe-4014-bcea-92bf42338c4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:28:14.082485 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef05f9c2-6efe-4014-bcea-92bf42338c4d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:14.089512 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1559710-cf15-4abd-b176-a0d229c306e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:14.090165 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1559710-cf15-4abd-b176-a0d229c306e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:14.090499 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1559710-cf15-4abd-b176-a0d229c306e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:14.090745 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1559710-cf15-4abd-b176-a0d229c306e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:14.101130 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1559710-cf15-4abd-b176-a0d229c306e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:14.102659 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1559710-cf15-4abd-b176-a0d229c306e5 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:14.112691 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1442/2856] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:28:14 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:28:14.477142 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ec6e91-7a20-475a-aee0-1f32cd1cf968 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:14.539277 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ec6e91-7a20-475a-aee0-1f32cd1cf968 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:14.543848 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477920e6-700c-472e-b464-dca5f1659183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:14.544706 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477920e6-700c-472e-b464-dca5f1659183 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:14.544983 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477920e6-700c-472e-b464-dca5f1659183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:14.545257 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477920e6-700c-472e-b464-dca5f1659183 admin admin] PATH_INFO: `/v3/users/de1f7ef8655744cf9244d3988a9fc04c` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:14.546424 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477920e6-700c-472e-b464-dca5f1659183 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de1f7ef8655744cf9244d3988a9fc04c)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:14.548357 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477920e6-700c-472e-b464-dca5f1659183 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:14.593784 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-477920e6-700c-472e-b464-dca5f1659183 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de1f7ef8655744cf9244d3988a9fc04c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:14.601196 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-477920e6-700c-472e-b464-dca5f1659183 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de1f7ef8655744cf9244d3988a9fc04c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:14.608168 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-477920e6-700c-472e-b464-dca5f1659183 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de1f7ef8655744cf9244d3988a9fc04c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:14.608799 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-477920e6-700c-472e-b464-dca5f1659183 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de1f7ef8655744cf9244d3988a9fc04c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:14.609405 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-477920e6-700c-472e-b464-dca5f1659183 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de1f7ef8655744cf9244d3988a9fc04c'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:14.626444 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1415/2857] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:28:14 2022] DELETE /identity/v3/users/de1f7ef8655744cf9244d3988a9fc04c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:28:14.631037 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6d738-ecfc-427f-9589-725b6936322f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:14.631325 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6d738-ecfc-427f-9589-725b6936322f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:14.631603 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6d738-ecfc-427f-9589-725b6936322f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:14.649663 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-28f6d738-ecfc-427f-9589-725b6936322f None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:28:14.650842 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-28f6d738-ecfc-427f-9589-725b6936322f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:28:14.710059 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1443/2858] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:28:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:28:14.720228 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477920e6-700c-472e-b464-dca5f1659183 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:14.721991 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-477920e6-700c-472e-b464-dca5f1659183 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:28:14.732839 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:28:14.780410 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477920e6-700c-472e-b464-dca5f1659183 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:14.784064 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e943f462-7e2e-415d-b9de-5ff2282aec87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:14.784709 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e943f462-7e2e-415d-b9de-5ff2282aec87 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:14.784994 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e943f462-7e2e-415d-b9de-5ff2282aec87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:14.785272 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e943f462-7e2e-415d-b9de-5ff2282aec87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:14.794285 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e943f462-7e2e-415d-b9de-5ff2282aec87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:14.795753 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e943f462-7e2e-415d-b9de-5ff2282aec87 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:14.805171 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1416/2859] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:28:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:28:15.082025 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f6d738-ecfc-427f-9589-725b6936322f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:15.086492 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-28f6d738-ecfc-427f-9589-725b6936322f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:28:15.106029 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f6d738-ecfc-427f-9589-725b6936322f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:15.110964 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f0cfe-b199-4f73-a35b-cc352407e353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:15.111860 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f0cfe-b199-4f73-a35b-cc352407e353 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:15.112143 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f0cfe-b199-4f73-a35b-cc352407e353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:15.112422 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f0cfe-b199-4f73-a35b-cc352407e353 admin admin] PATH_INFO: `/v3/projects/cc216780212c4dfa880d7ccc439f416f` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:15.113610 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275f0cfe-b199-4f73-a35b-cc352407e353 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc216780212c4dfa880d7ccc439f416f)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:15.117268 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275f0cfe-b199-4f73-a35b-cc352407e353 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:15.142024 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:28:15.169817 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-275f0cfe-b199-4f73-a35b-cc352407e353 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc216780212c4dfa880d7ccc439f416f', 'request_id': 'req-275f0cfe-b199-4f73-a35b-cc352407e353'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:15.178194 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-275f0cfe-b199-4f73-a35b-cc352407e353 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc216780212c4dfa880d7ccc439f416f', 'request_id': 'req-275f0cfe-b199-4f73-a35b-cc352407e353'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:15.183779 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-275f0cfe-b199-4f73-a35b-cc352407e353 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc216780212c4dfa880d7ccc439f416f', 'request_id': 'req-275f0cfe-b199-4f73-a35b-cc352407e353'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:15.196820 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-275f0cfe-b199-4f73-a35b-cc352407e353 admin admin] The token cache is being invalidate because project cc216780212c4dfa880d7ccc439f416f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:28:15.197221 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-275f0cfe-b199-4f73-a35b-cc352407e353 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:15.198157 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-275f0cfe-b199-4f73-a35b-cc352407e353 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:15.200347 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1444/2860] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:28:15 2022] DELETE /identity/v3/projects/cc216780212c4dfa880d7ccc439f416f => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:28:15.204341 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e943f462-7e2e-415d-b9de-5ff2282aec87 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:15.214282 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e943f462-7e2e-415d-b9de-5ff2282aec87 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:15.271462 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80918c87-c783-4e96-a434-3ba70f064390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:15.272439 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80918c87-c783-4e96-a434-3ba70f064390 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:15.272721 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80918c87-c783-4e96-a434-3ba70f064390 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:15.273038 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80918c87-c783-4e96-a434-3ba70f064390 admin admin] PATH_INFO: `/v3/users/a75ee2bc9d2c4e6a82fc46217970945d` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:15.274359 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80918c87-c783-4e96-a434-3ba70f064390 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a75ee2bc9d2c4e6a82fc46217970945d)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:15.276490 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80918c87-c783-4e96-a434-3ba70f064390 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:15.323899 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-80918c87-c783-4e96-a434-3ba70f064390 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a75ee2bc9d2c4e6a82fc46217970945d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:15.331355 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-80918c87-c783-4e96-a434-3ba70f064390 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a75ee2bc9d2c4e6a82fc46217970945d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:15.338866 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-80918c87-c783-4e96-a434-3ba70f064390 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a75ee2bc9d2c4e6a82fc46217970945d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:15.339523 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-80918c87-c783-4e96-a434-3ba70f064390 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a75ee2bc9d2c4e6a82fc46217970945d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:15.340136 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-80918c87-c783-4e96-a434-3ba70f064390 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a75ee2bc9d2c4e6a82fc46217970945d'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:15.358490 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1417/2861] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:28:15 2022] DELETE /identity/v3/users/a75ee2bc9d2c4e6a82fc46217970945d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:28:15.898123 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f0cfe-b199-4f73-a35b-cc352407e353 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:15.899631 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-275f0cfe-b199-4f73-a35b-cc352407e353 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:28:15.917105 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f0cfe-b199-4f73-a35b-cc352407e353 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:15.975304 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66faf1c4-4885-41ec-b497-6bc7aaf22025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:15.976172 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66faf1c4-4885-41ec-b497-6bc7aaf22025 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:15.976455 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66faf1c4-4885-41ec-b497-6bc7aaf22025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:15.976733 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66faf1c4-4885-41ec-b497-6bc7aaf22025 admin admin] PATH_INFO: `/v3/projects/b96259df88e44bd6ae04e3f54202618d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:15.977917 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66faf1c4-4885-41ec-b497-6bc7aaf22025 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b96259df88e44bd6ae04e3f54202618d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:15.979865 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66faf1c4-4885-41ec-b497-6bc7aaf22025 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:16.029253 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-66faf1c4-4885-41ec-b497-6bc7aaf22025 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b96259df88e44bd6ae04e3f54202618d', 'request_id': 'req-66faf1c4-4885-41ec-b497-6bc7aaf22025'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:16.038053 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-66faf1c4-4885-41ec-b497-6bc7aaf22025 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b96259df88e44bd6ae04e3f54202618d', 'request_id': 'req-66faf1c4-4885-41ec-b497-6bc7aaf22025'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:16.045122 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-66faf1c4-4885-41ec-b497-6bc7aaf22025 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b96259df88e44bd6ae04e3f54202618d', 'request_id': 'req-66faf1c4-4885-41ec-b497-6bc7aaf22025'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:16.061761 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-66faf1c4-4885-41ec-b497-6bc7aaf22025 admin admin] The token cache is being invalidate because project b96259df88e44bd6ae04e3f54202618d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:28:16.062185 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-66faf1c4-4885-41ec-b497-6bc7aaf22025 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:16.063125 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-66faf1c4-4885-41ec-b497-6bc7aaf22025 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:16.065300 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1445/2862] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:28:15 2022] DELETE /identity/v3/projects/b96259df88e44bd6ae04e3f54202618d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:28:16.072803 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3800edf2-955b-4eb8-8110-85d09520f0e4 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:16.073383 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3800edf2-955b-4eb8-8110-85d09520f0e4 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:16.073383 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3800edf2-955b-4eb8-8110-85d09520f0e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:16.101722 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-3800edf2-955b-4eb8-8110-85d09520f0e4 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:28:16.101722 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-3800edf2-955b-4eb8-8110-85d09520f0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:28:16.128780 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:28:16.174322 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1418/2863] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:28:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:28:16.182400 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e6758-4a5d-4b21-993f-474844106717 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:16.182904 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e6758-4a5d-4b21-993f-474844106717 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:16.183112 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e6758-4a5d-4b21-993f-474844106717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:16.201861 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3d9e6758-4a5d-4b21-993f-474844106717 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:28:16.203091 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3d9e6758-4a5d-4b21-993f-474844106717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:28:16.211089 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1446/2864] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:28:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:28:16.215108 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3800edf2-955b-4eb8-8110-85d09520f0e4 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:16.225595 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3800edf2-955b-4eb8-8110-85d09520f0e4 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:16.230200 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c39f0a-72c8-4e8b-8e93-62c0b42433bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:16.230867 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c39f0a-72c8-4e8b-8e93-62c0b42433bd admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:16.231174 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c39f0a-72c8-4e8b-8e93-62c0b42433bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:16.231484 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c39f0a-72c8-4e8b-8e93-62c0b42433bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:16.232215 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c39f0a-72c8-4e8b-8e93-62c0b42433bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:28:16.232526 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c39f0a-72c8-4e8b-8e93-62c0b42433bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:16.234137 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c39f0a-72c8-4e8b-8e93-62c0b42433bd admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:16.247595 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1419/2865] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:28:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:28:16.251979 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9e6758-4a5d-4b21-993f-474844106717 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:16.261332 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9e6758-4a5d-4b21-993f-474844106717 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:16.265993 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69012105-0d30-49c3-84b5-c55df9474b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:16.266648 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69012105-0d30-49c3-84b5-c55df9474b0b admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:16.266959 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69012105-0d30-49c3-84b5-c55df9474b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:16.267378 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69012105-0d30-49c3-84b5-c55df9474b0b admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:16.268078 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69012105-0d30-49c3-84b5-c55df9474b0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:16.269844 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69012105-0d30-49c3-84b5-c55df9474b0b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:16.296014 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1447/2866] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:28:16 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:28:16.300215 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c39f0a-72c8-4e8b-8e93-62c0b42433bd admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:16.311243 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c39f0a-72c8-4e8b-8e93-62c0b42433bd admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:16.369516 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86040868-4f42-4bfe-9e78-46b205299950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:16.370083 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86040868-4f42-4bfe-9e78-46b205299950 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:16.370362 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86040868-4f42-4bfe-9e78-46b205299950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:16.370635 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86040868-4f42-4bfe-9e78-46b205299950 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:16.371346 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86040868-4f42-4bfe-9e78-46b205299950 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:16.373257 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86040868-4f42-4bfe-9e78-46b205299950 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:16.393035 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1420/2867] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:28:16 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:28:16.397863 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69012105-0d30-49c3-84b5-c55df9474b0b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:16.408208 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69012105-0d30-49c3-84b5-c55df9474b0b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:16.413372 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51af9f5f-dd97-431d-be9d-5778e7a12078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:16.414056 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af9f5f-dd97-431d-be9d-5778e7a12078 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:16.414318 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af9f5f-dd97-431d-be9d-5778e7a12078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:16.414625 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af9f5f-dd97-431d-be9d-5778e7a12078 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:16.415363 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51af9f5f-dd97-431d-be9d-5778e7a12078 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:16.416946 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51af9f5f-dd97-431d-be9d-5778e7a12078 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:16.419050 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:28:16.429396 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1448/2868] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:28:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:28:16.433516 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86040868-4f42-4bfe-9e78-46b205299950 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:16.444472 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86040868-4f42-4bfe-9e78-46b205299950 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:16.458830 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f2c1a0-9877-4c6a-91de-aa70a42a76d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:16.459572 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f2c1a0-9877-4c6a-91de-aa70a42a76d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:16.459902 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f2c1a0-9877-4c6a-91de-aa70a42a76d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:16.460223 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f2c1a0-9877-4c6a-91de-aa70a42a76d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:16.460997 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f2c1a0-9877-4c6a-91de-aa70a42a76d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:16.462667 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f2c1a0-9877-4c6a-91de-aa70a42a76d6 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:16.475288 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1421/2869] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:28:16 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:28:16.479740 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51af9f5f-dd97-431d-be9d-5778e7a12078 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:16.492764 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51af9f5f-dd97-431d-be9d-5778e7a12078 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:16.498453 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c078da59-8b7c-409d-a2ad-99f01a5224d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:16.499599 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c078da59-8b7c-409d-a2ad-99f01a5224d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:16.499924 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c078da59-8b7c-409d-a2ad-99f01a5224d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:16.500278 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c078da59-8b7c-409d-a2ad-99f01a5224d7 admin admin] PATH_INFO: `/v3/projects/e185bbc75b474c2bb4f6424cbb0efdc7/users/541bd7b41d004fd9903cff1bfc1dbc54/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:16.518388 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c078da59-8b7c-409d-a2ad-99f01a5224d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e185bbc75b474c2bb4f6424cbb0efdc7, user_id=541bd7b41d004fd9903cff1bfc1dbc54, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:16.521837 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c078da59-8b7c-409d-a2ad-99f01a5224d7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:16.537645 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1449/2870] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:28:16 2022] PUT /identity/v3/projects/e185bbc75b474c2bb4f6424cbb0efdc7/users/541bd7b41d004fd9903cff1bfc1dbc54/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:28:16.551267 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52571d9-6b66-4799-a2ba-76c42f996057 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:16.552103 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52571d9-6b66-4799-a2ba-76c42f996057 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:16.552393 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52571d9-6b66-4799-a2ba-76c42f996057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:16.578380 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-f52571d9-6b66-4799-a2ba-76c42f996057 None None] MFA Rules not processed for user `541bd7b41d004fd9903cff1bfc1dbc54`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:28:16.579608 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f52571d9-6b66-4799-a2ba-76c42f996057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:28:16.646604 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1422/2871] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:28:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:28:16.651046 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c078da59-8b7c-409d-a2ad-99f01a5224d7 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:16.713412 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c078da59-8b7c-409d-a2ad-99f01a5224d7 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:16.718289 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152dc1e6-3126-46f8-b7b8-781bc19d5397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:16.718976 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152dc1e6-3126-46f8-b7b8-781bc19d5397 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:16.719266 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152dc1e6-3126-46f8-b7b8-781bc19d5397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:16.719542 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152dc1e6-3126-46f8-b7b8-781bc19d5397 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:16.720231 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152dc1e6-3126-46f8-b7b8-781bc19d5397 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:16.722474 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152dc1e6-3126-46f8-b7b8-781bc19d5397 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:16.748500 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1450/2872] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:28:16 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:28:16.752536 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52571d9-6b66-4799-a2ba-76c42f996057 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:16.763350 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52571d9-6b66-4799-a2ba-76c42f996057 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:16.788049 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:28:16.820500 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1e0e69-687b-49bc-86e4-fc07cc6177c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:16.821165 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e0e69-687b-49bc-86e4-fc07cc6177c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:16.821444 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e0e69-687b-49bc-86e4-fc07cc6177c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:16.821717 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e0e69-687b-49bc-86e4-fc07cc6177c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:16.822394 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1e0e69-687b-49bc-86e4-fc07cc6177c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:16.824356 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1e0e69-687b-49bc-86e4-fc07cc6177c3 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:16.844608 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1423/2873] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:28:16 2022] POST /identity/v3/users => generated 299 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:28:16.849592 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152dc1e6-3126-46f8-b7b8-781bc19d5397 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:16.859903 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152dc1e6-3126-46f8-b7b8-781bc19d5397 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:16.865066 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7789af-779c-4251-8a16-abf4939310f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:16.865724 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7789af-779c-4251-8a16-abf4939310f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:16.866005 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7789af-779c-4251-8a16-abf4939310f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:16.866307 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7789af-779c-4251-8a16-abf4939310f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:16.867066 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7789af-779c-4251-8a16-abf4939310f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:16.868665 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7789af-779c-4251-8a16-abf4939310f9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:16.879190 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1451/2874] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:28:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:28:16.883878 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1e0e69-687b-49bc-86e4-fc07cc6177c3 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:16.893668 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1e0e69-687b-49bc-86e4-fc07cc6177c3 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:16.899434 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492f630f-eb21-44f6-9a5a-6d1df1955678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:16.900359 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f630f-eb21-44f6-9a5a-6d1df1955678 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:16.900659 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f630f-eb21-44f6-9a5a-6d1df1955678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:16.900947 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f630f-eb21-44f6-9a5a-6d1df1955678 admin admin] PATH_INFO: `/v3/projects/9bd29fb76ea84da185427137c3042433/users/0ed452f651744917adbe9da06df79c07/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:16.916971 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492f630f-eb21-44f6-9a5a-6d1df1955678 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bd29fb76ea84da185427137c3042433, user_id=0ed452f651744917adbe9da06df79c07, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:16.919652 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492f630f-eb21-44f6-9a5a-6d1df1955678 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:16.933361 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1424/2875] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:28:16 2022] PUT /identity/v3/projects/9bd29fb76ea84da185427137c3042433/users/0ed452f651744917adbe9da06df79c07/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:28:16.937712 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7789af-779c-4251-8a16-abf4939310f9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:16.948939 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7789af-779c-4251-8a16-abf4939310f9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:17.007427 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f399c1f5-c64c-4127-aa4e-65f42bb20c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:17.008099 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399c1f5-c64c-4127-aa4e-65f42bb20c85 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:17.008387 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399c1f5-c64c-4127-aa4e-65f42bb20c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:17.008669 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399c1f5-c64c-4127-aa4e-65f42bb20c85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:17.009392 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f399c1f5-c64c-4127-aa4e-65f42bb20c85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:17.011006 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f399c1f5-c64c-4127-aa4e-65f42bb20c85 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:17.022313 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1452/2876] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:28:16 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:28:17.026499 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492f630f-eb21-44f6-9a5a-6d1df1955678 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:17.037535 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492f630f-eb21-44f6-9a5a-6d1df1955678 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:17.042133 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f64ab56-4b73-4544-998a-5da648197915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:17.043049 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f64ab56-4b73-4544-998a-5da648197915 admin admin] REQUEST_METHOD: `PUT` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:17.043335 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f64ab56-4b73-4544-998a-5da648197915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:17.043612 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f64ab56-4b73-4544-998a-5da648197915 admin admin] PATH_INFO: `/v3/projects/9bd29fb76ea84da185427137c3042433/users/0ed452f651744917adbe9da06df79c07/roles/759d845eb6284e42931b4265af448ebc` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:17.045635 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f64ab56-4b73-4544-998a-5da648197915 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bd29fb76ea84da185427137c3042433, user_id=0ed452f651744917adbe9da06df79c07, role_id=759d845eb6284e42931b4265af448ebc)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:17.048346 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f64ab56-4b73-4544-998a-5da648197915 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:17.059122 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1425/2877] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:28:17 2022] PUT /identity/v3/projects/9bd29fb76ea84da185427137c3042433/users/0ed452f651744917adbe9da06df79c07/roles/759d845eb6284e42931b4265af448ebc => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:28:17.070144 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d035f8-016a-44a4-95a5-49e5cb916112 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:17.070438 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d035f8-016a-44a4-95a5-49e5cb916112 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:17.070756 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d035f8-016a-44a4-95a5-49e5cb916112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:17.099429 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-53d035f8-016a-44a4-95a5-49e5cb916112 None None] MFA Rules not processed for user `0ed452f651744917adbe9da06df79c07`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:28:17.100644 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-53d035f8-016a-44a4-95a5-49e5cb916112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:28:17.133381 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:28:17.171469 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1453/2878] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:28:17 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:28:17.182498 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f64ab56-4b73-4544-998a-5da648197915 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:17.184336 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-9f64ab56-4b73-4544-998a-5da648197915 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:28:17.244375 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f64ab56-4b73-4544-998a-5da648197915 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:17.248330 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cda500-b204-4597-8f8f-99ffe6ca57b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:17.248979 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cda500-b204-4597-8f8f-99ffe6ca57b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:17.249263 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cda500-b204-4597-8f8f-99ffe6ca57b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:17.249543 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cda500-b204-4597-8f8f-99ffe6ca57b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:17.306436 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cda500-b204-4597-8f8f-99ffe6ca57b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:17.307921 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cda500-b204-4597-8f8f-99ffe6ca57b1 service cinder] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:17.316783 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1426/2879] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:28:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:28:21.552217 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d035f8-016a-44a4-95a5-49e5cb916112 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:21.554962 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-53d035f8-016a-44a4-95a5-49e5cb916112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:28:21.567410 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d035f8-016a-44a4-95a5-49e5cb916112 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:21.572692 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20183d09-e715-4bab-b1d7-54fbe98201de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:21.573356 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20183d09-e715-4bab-b1d7-54fbe98201de service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:21.573643 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20183d09-e715-4bab-b1d7-54fbe98201de service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:21.573912 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20183d09-e715-4bab-b1d7-54fbe98201de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:21.588191 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20183d09-e715-4bab-b1d7-54fbe98201de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:21.589628 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20183d09-e715-4bab-b1d7-54fbe98201de service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:21.600313 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1454/2880] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:28:21 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:28:26.553389 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cda500-b204-4597-8f8f-99ffe6ca57b1 service cinder] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:26.590043 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:28:26.623086 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cda500-b204-4597-8f8f-99ffe6ca57b1 service cinder] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:26.627837 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1292427-d882-437c-8c8d-dd85fb8a483e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:26.628700 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1292427-d882-437c-8c8d-dd85fb8a483e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:26.628982 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1292427-d882-437c-8c8d-dd85fb8a483e admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:26.629889 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1292427-d882-437c-8c8d-dd85fb8a483e admin admin] PATH_INFO: `/v3/users/541bd7b41d004fd9903cff1bfc1dbc54` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:26.631154 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1292427-d882-437c-8c8d-dd85fb8a483e admin admin] RBAC: Authorizing `identity:delete_user(user_id=541bd7b41d004fd9903cff1bfc1dbc54)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:26.633129 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1292427-d882-437c-8c8d-dd85fb8a483e admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:26.686064 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d1292427-d882-437c-8c8d-dd85fb8a483e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '541bd7b41d004fd9903cff1bfc1dbc54'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:26.695402 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d1292427-d882-437c-8c8d-dd85fb8a483e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '541bd7b41d004fd9903cff1bfc1dbc54'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:26.703050 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d1292427-d882-437c-8c8d-dd85fb8a483e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '541bd7b41d004fd9903cff1bfc1dbc54'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:26.703715 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d1292427-d882-437c-8c8d-dd85fb8a483e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '541bd7b41d004fd9903cff1bfc1dbc54'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:26.704314 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-d1292427-d882-437c-8c8d-dd85fb8a483e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '541bd7b41d004fd9903cff1bfc1dbc54'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:26.722564 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1427/2881] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:28:26 2022] DELETE /identity/v3/users/541bd7b41d004fd9903cff1bfc1dbc54 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:28:26.728197 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98797e-3891-429c-acd1-99fd3dd98018 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:26.728677 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98797e-3891-429c-acd1-99fd3dd98018 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:26.728779 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98797e-3891-429c-acd1-99fd3dd98018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:26.746515 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-cd98797e-3891-429c-acd1-99fd3dd98018 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:28:26.747717 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cd98797e-3891-429c-acd1-99fd3dd98018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:28:26.813008 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1455/2882] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:28:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:28:26.822957 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1292427-d882-437c-8c8d-dd85fb8a483e admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:26.824501 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-d1292427-d882-437c-8c8d-dd85fb8a483e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:28:26.886753 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1292427-d882-437c-8c8d-dd85fb8a483e admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:26.890772 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d6348e-6ddf-42ba-91a6-bc50269779f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:26.891590 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d6348e-6ddf-42ba-91a6-bc50269779f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:26.891881 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d6348e-6ddf-42ba-91a6-bc50269779f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:26.892166 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d6348e-6ddf-42ba-91a6-bc50269779f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:26.902312 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d6348e-6ddf-42ba-91a6-bc50269779f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:26.903798 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d6348e-6ddf-42ba-91a6-bc50269779f2 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:26.913726 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1428/2883] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:28:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:28:27.197171 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd98797e-3891-429c-acd1-99fd3dd98018 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:27.199060 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cd98797e-3891-429c-acd1-99fd3dd98018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:28:27.213167 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd98797e-3891-429c-acd1-99fd3dd98018 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:27.218008 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9891a2b0-fe5b-4209-949c-91a2ecaf3e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:27.218907 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891a2b0-fe5b-4209-949c-91a2ecaf3e1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:27.219199 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891a2b0-fe5b-4209-949c-91a2ecaf3e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:27.219473 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891a2b0-fe5b-4209-949c-91a2ecaf3e1e admin admin] PATH_INFO: `/v3/projects/e185bbc75b474c2bb4f6424cbb0efdc7` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:27.221043 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9891a2b0-fe5b-4209-949c-91a2ecaf3e1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e185bbc75b474c2bb4f6424cbb0efdc7)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:27.224212 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9891a2b0-fe5b-4209-949c-91a2ecaf3e1e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:27.279366 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9891a2b0-fe5b-4209-949c-91a2ecaf3e1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e185bbc75b474c2bb4f6424cbb0efdc7', 'request_id': 'req-9891a2b0-fe5b-4209-949c-91a2ecaf3e1e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:27.289667 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9891a2b0-fe5b-4209-949c-91a2ecaf3e1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e185bbc75b474c2bb4f6424cbb0efdc7', 'request_id': 'req-9891a2b0-fe5b-4209-949c-91a2ecaf3e1e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:27.294200 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9891a2b0-fe5b-4209-949c-91a2ecaf3e1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e185bbc75b474c2bb4f6424cbb0efdc7', 'request_id': 'req-9891a2b0-fe5b-4209-949c-91a2ecaf3e1e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:27.311919 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9891a2b0-fe5b-4209-949c-91a2ecaf3e1e admin admin] The token cache is being invalidate because project e185bbc75b474c2bb4f6424cbb0efdc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:28:27.312214 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9891a2b0-fe5b-4209-949c-91a2ecaf3e1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:27.312881 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9891a2b0-fe5b-4209-949c-91a2ecaf3e1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:27.314494 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1456/2884] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:28:27 2022] DELETE /identity/v3/projects/e185bbc75b474c2bb4f6424cbb0efdc7 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:28:27.319069 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d6348e-6ddf-42ba-91a6-bc50269779f2 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:27.329875 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d6348e-6ddf-42ba-91a6-bc50269779f2 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:27.384665 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:28:27.392600 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff4e4cd-860e-47d7-9ea7-7a1bec85ac21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:27.393644 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff4e4cd-860e-47d7-9ea7-7a1bec85ac21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:27.393945 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff4e4cd-860e-47d7-9ea7-7a1bec85ac21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:27.394442 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff4e4cd-860e-47d7-9ea7-7a1bec85ac21 admin admin] PATH_INFO: `/v3/users/0ed452f651744917adbe9da06df79c07` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:27.396466 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff4e4cd-860e-47d7-9ea7-7a1bec85ac21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ed452f651744917adbe9da06df79c07)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:27.398816 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff4e4cd-860e-47d7-9ea7-7a1bec85ac21 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:27.447851 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1ff4e4cd-860e-47d7-9ea7-7a1bec85ac21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ed452f651744917adbe9da06df79c07'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:27.455750 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1ff4e4cd-860e-47d7-9ea7-7a1bec85ac21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ed452f651744917adbe9da06df79c07'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:27.463930 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1ff4e4cd-860e-47d7-9ea7-7a1bec85ac21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ed452f651744917adbe9da06df79c07'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:27.464632 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1ff4e4cd-860e-47d7-9ea7-7a1bec85ac21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ed452f651744917adbe9da06df79c07'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:27.465247 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-1ff4e4cd-860e-47d7-9ea7-7a1bec85ac21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ed452f651744917adbe9da06df79c07'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:27.484704 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1429/2885] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:28:27 2022] DELETE /identity/v3/users/0ed452f651744917adbe9da06df79c07 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:28:27.781841 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9891a2b0-fe5b-4209-949c-91a2ecaf3e1e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:27.783691 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9891a2b0-fe5b-4209-949c-91a2ecaf3e1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:28:27.800772 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9891a2b0-fe5b-4209-949c-91a2ecaf3e1e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:27.851566 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:28:27.874461 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c1f5f3-2aeb-461d-a1bc-46ec4414cbb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:27.875379 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1f5f3-2aeb-461d-a1bc-46ec4414cbb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:27.875665 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1f5f3-2aeb-461d-a1bc-46ec4414cbb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:27.875943 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c1f5f3-2aeb-461d-a1bc-46ec4414cbb6 admin admin] PATH_INFO: `/v3/projects/9bd29fb76ea84da185427137c3042433` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:27.877169 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c1f5f3-2aeb-461d-a1bc-46ec4414cbb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bd29fb76ea84da185427137c3042433)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:27.879352 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c1f5f3-2aeb-461d-a1bc-46ec4414cbb6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:27.933450 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f5c1f5f3-2aeb-461d-a1bc-46ec4414cbb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bd29fb76ea84da185427137c3042433', 'request_id': 'req-f5c1f5f3-2aeb-461d-a1bc-46ec4414cbb6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:27.940974 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f5c1f5f3-2aeb-461d-a1bc-46ec4414cbb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bd29fb76ea84da185427137c3042433', 'request_id': 'req-f5c1f5f3-2aeb-461d-a1bc-46ec4414cbb6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:27.947451 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f5c1f5f3-2aeb-461d-a1bc-46ec4414cbb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bd29fb76ea84da185427137c3042433', 'request_id': 'req-f5c1f5f3-2aeb-461d-a1bc-46ec4414cbb6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:27.959710 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f5c1f5f3-2aeb-461d-a1bc-46ec4414cbb6 admin admin] The token cache is being invalidate because project 9bd29fb76ea84da185427137c3042433 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:28:27.960002 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f5c1f5f3-2aeb-461d-a1bc-46ec4414cbb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:27.960657 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f5c1f5f3-2aeb-461d-a1bc-46ec4414cbb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:27.962202 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1457/2886] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:28:27 2022] DELETE /identity/v3/projects/9bd29fb76ea84da185427137c3042433 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:28:27.969202 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ed0b2-5d40-4e0b-8194-467540ff2750 None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:27.969683 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ed0b2-5d40-4e0b-8194-467540ff2750 None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:27.969842 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ed0b2-5d40-4e0b-8194-467540ff2750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:27.989920 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-960ed0b2-5d40-4e0b-8194-467540ff2750 None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:28:27.991162 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-960ed0b2-5d40-4e0b-8194-467540ff2750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:28:28.063475 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1430/2887] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:28:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:28:28.074753 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c521c-e83e-479c-b130-d448c5bd9d1b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:28.075089 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c521c-e83e-479c-b130-d448c5bd9d1b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:28.075377 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4c521c-e83e-479c-b130-d448c5bd9d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:28.094645 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2d4c521c-e83e-479c-b130-d448c5bd9d1b None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:28:28.096036 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2d4c521c-e83e-479c-b130-d448c5bd9d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:28:28.104696 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1458/2888] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:28:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 27 15:28:28.109244 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ed0b2-5d40-4e0b-8194-467540ff2750 None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:28.119509 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ed0b2-5d40-4e0b-8194-467540ff2750 None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:28.125188 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1bbee1-8c2a-4415-b78f-22f3fed282f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:28.125833 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1bbee1-8c2a-4415-b78f-22f3fed282f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:28.126117 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1bbee1-8c2a-4415-b78f-22f3fed282f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:28.126393 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1bbee1-8c2a-4415-b78f-22f3fed282f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:28.127185 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1bbee1-8c2a-4415-b78f-22f3fed282f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80394) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 27 15:28:28.127468 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1bbee1-8c2a-4415-b78f-22f3fed282f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:28.129742 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1bbee1-8c2a-4415-b78f-22f3fed282f8 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:28.141316 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1431/2889] 10.222.0.50 () {58 vars in 1152 bytes} [Fri May 27 15:28:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 27 15:28:28.146150 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c521c-e83e-479c-b130-d448c5bd9d1b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:28.156144 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4c521c-e83e-479c-b130-d448c5bd9d1b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:28.160559 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74567916-0bad-431f-9519-72f50e269b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:28.161560 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74567916-0bad-431f-9519-72f50e269b46 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:28.161903 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74567916-0bad-431f-9519-72f50e269b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:28.162124 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74567916-0bad-431f-9519-72f50e269b46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:28.162857 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74567916-0bad-431f-9519-72f50e269b46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:28.164823 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74567916-0bad-431f-9519-72f50e269b46 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:28.195106 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1459/2890] 10.222.0.50 () {60 vars in 1152 bytes} [Fri May 27 15:28:28 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:28:28.199450 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1bbee1-8c2a-4415-b78f-22f3fed282f8 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:28.210621 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1bbee1-8c2a-4415-b78f-22f3fed282f8 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:28.271165 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edf7dc9-bce3-483c-840e-2f4379d0288b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:28.271836 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf7dc9-bce3-483c-840e-2f4379d0288b admin admin] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:28.272129 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf7dc9-bce3-483c-840e-2f4379d0288b admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:28.272398 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf7dc9-bce3-483c-840e-2f4379d0288b admin admin] PATH_INFO: `/v3/users` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:28.273090 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edf7dc9-bce3-483c-840e-2f4379d0288b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:28.275038 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edf7dc9-bce3-483c-840e-2f4379d0288b admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:28.282876 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:28:28.298809 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1432/2891] 10.222.0.50 () {60 vars in 1143 bytes} [Fri May 27 15:28:28 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 27 15:28:28.300052 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74567916-0bad-431f-9519-72f50e269b46 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:28.313516 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74567916-0bad-431f-9519-72f50e269b46 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:28.319317 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8672b199-f702-4c21-93c8-843d75f77b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:28.320016 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8672b199-f702-4c21-93c8-843d75f77b25 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:28.320300 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8672b199-f702-4c21-93c8-843d75f77b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:28.320584 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8672b199-f702-4c21-93c8-843d75f77b25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:28.321331 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8672b199-f702-4c21-93c8-843d75f77b25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:28.323564 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8672b199-f702-4c21-93c8-843d75f77b25 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:28.339449 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1460/2892] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:28:28 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 27 15:28:28.343270 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edf7dc9-bce3-483c-840e-2f4379d0288b admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:28.355003 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edf7dc9-bce3-483c-840e-2f4379d0288b admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:28.361433 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2511e72-e518-4dd3-b350-da4881621dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:28.362361 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2511e72-e518-4dd3-b350-da4881621dcb admin admin] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:28.362649 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2511e72-e518-4dd3-b350-da4881621dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:28.362969 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2511e72-e518-4dd3-b350-da4881621dcb admin admin] PATH_INFO: `/v3/roles` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:28.364750 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2511e72-e518-4dd3-b350-da4881621dcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:28.366950 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2511e72-e518-4dd3-b350-da4881621dcb admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:28.380753 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1433/2893] 10.222.0.50 () {58 vars in 1121 bytes} [Fri May 27 15:28:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 27 15:28:28.384981 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8672b199-f702-4c21-93c8-843d75f77b25 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:28.395897 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8672b199-f702-4c21-93c8-843d75f77b25 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:28.400592 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9198d27-0a9a-42e3-b39d-f2fb4f0435c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:28.401482 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9198d27-0a9a-42e3-b39d-f2fb4f0435c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:28.401770 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9198d27-0a9a-42e3-b39d-f2fb4f0435c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:28.402050 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9198d27-0a9a-42e3-b39d-f2fb4f0435c8 admin admin] PATH_INFO: `/v3/projects/8031cc1037dd48ebbf72c9bcc8ef46ba/users/ea0a719c3a164eb2a3dba0ca6c964fc0/roles/8442c42e847a4b38b76f3a7fbe9ada85` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:28.419934 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9198d27-0a9a-42e3-b39d-f2fb4f0435c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8031cc1037dd48ebbf72c9bcc8ef46ba, user_id=ea0a719c3a164eb2a3dba0ca6c964fc0, role_id=8442c42e847a4b38b76f3a7fbe9ada85)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:28.422586 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9198d27-0a9a-42e3-b39d-f2fb4f0435c8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:28.435939 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1461/2894] 10.222.0.50 () {60 vars in 1482 bytes} [Fri May 27 15:28:28 2022] PUT /identity/v3/projects/8031cc1037dd48ebbf72c9bcc8ef46ba/users/ea0a719c3a164eb2a3dba0ca6c964fc0/roles/8442c42e847a4b38b76f3a7fbe9ada85 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:28:28.448614 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ca2ff-6b28-48f5-92c5-bebba330d39a None None] REQUEST_METHOD: `POST` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:28.448907 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ca2ff-6b28-48f5-92c5-bebba330d39a None None] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:28.449190 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ca2ff-6b28-48f5-92c5-bebba330d39a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:28.477802 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.auth.core [None req-404ca2ff-6b28-48f5-92c5-bebba330d39a None None] MFA Rules not processed for user `ea0a719c3a164eb2a3dba0ca6c964fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80394) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:28:28.479096 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-404ca2ff-6b28-48f5-92c5-bebba330d39a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:28:28.553146 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1434/2895] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:28:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:28:28.567939 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9198d27-0a9a-42e3-b39d-f2fb4f0435c8 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:28.569681 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e9198d27-0a9a-42e3-b39d-f2fb4f0435c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:28:28.633065 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9198d27-0a9a-42e3-b39d-f2fb4f0435c8 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:28.636727 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddb43fb-ea44-4a3b-a297-63e22615f8ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '896af6ccf89847e5903f6a4fe3d9d810', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:28.637399 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb43fb-ea44-4a3b-a297-63e22615f8ab service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:28.637677 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb43fb-ea44-4a3b-a297-63e22615f8ab service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:28.637952 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddb43fb-ea44-4a3b-a297-63e22615f8ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:28.648003 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddb43fb-ea44-4a3b-a297-63e22615f8ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:28.649442 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddb43fb-ea44-4a3b-a297-63e22615f8ab service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:28.659184 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1462/2896] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:28:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:28:36.080331 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404ca2ff-6b28-48f5-92c5-bebba330d39a None None] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:36.143297 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404ca2ff-6b28-48f5-92c5-bebba330d39a None None] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:36.148597 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f2d44-fe51-43a6-999e-db72a5428511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:36.149479 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f2d44-fe51-43a6-999e-db72a5428511 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:36.149754 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f2d44-fe51-43a6-999e-db72a5428511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:36.150033 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f2d44-fe51-43a6-999e-db72a5428511 admin admin] PATH_INFO: `/v3/users/ea0a719c3a164eb2a3dba0ca6c964fc0` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:36.151297 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63f2d44-fe51-43a6-999e-db72a5428511 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea0a719c3a164eb2a3dba0ca6c964fc0)` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:36.153240 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63f2d44-fe51-43a6-999e-db72a5428511 admin admin] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:36.204063 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a63f2d44-fe51-43a6-999e-db72a5428511 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea0a719c3a164eb2a3dba0ca6c964fc0'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:36.212958 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a63f2d44-fe51-43a6-999e-db72a5428511 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea0a719c3a164eb2a3dba0ca6c964fc0'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:36.221070 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a63f2d44-fe51-43a6-999e-db72a5428511 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea0a719c3a164eb2a3dba0ca6c964fc0'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:36.222000 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a63f2d44-fe51-43a6-999e-db72a5428511 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea0a719c3a164eb2a3dba0ca6c964fc0'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:36.223293 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.notifications [None req-a63f2d44-fe51-43a6-999e-db72a5428511 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea0a719c3a164eb2a3dba0ca6c964fc0'} {{(pid=80394) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:36.232152 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:INSERT=3,keystone:SELECT=88,keystone:DELETE=9 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:28:36.245810 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1435/2897] 10.222.0.50 () {58 vars in 1223 bytes} [Fri May 27 15:28:36 2022] DELETE /identity/v3/users/ea0a719c3a164eb2a3dba0ca6c964fc0 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:28:36.252515 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6ccc2-a556-4c2c-91bc-7a98cb25ad6f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:36.253005 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6ccc2-a556-4c2c-91bc-7a98cb25ad6f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:36.253118 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6ccc2-a556-4c2c-91bc-7a98cb25ad6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:36.272447 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-79c6ccc2-a556-4c2c-91bc-7a98cb25ad6f None None] MFA Rules not processed for user `9c29995c0b564cd7879c58c75dc099cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 27 15:28:36.273651 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-79c6ccc2-a556-4c2c-91bc-7a98cb25ad6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:28:36.277693 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:28:36.340783 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1463/2898] 10.222.0.50 () {58 vars in 957 bytes} [Fri May 27 15:28:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 27 15:28:36.351421 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f2d44-fe51-43a6-999e-db72a5428511 admin admin] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:36.353297 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-a63f2d44-fe51-43a6-999e-db72a5428511 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:28:36.424111 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63f2d44-fe51-43a6-999e-db72a5428511 admin admin] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:36.429081 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d63f06-9353-4ed5-bd18-2a7ef5c75ad0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f874edc08438434189f1d14d79291487', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:36.429731 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d63f06-9353-4ed5-bd18-2a7ef5c75ad0 service neutron] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:36.430015 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d63f06-9353-4ed5-bd18-2a7ef5c75ad0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:36.430294 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d63f06-9353-4ed5-bd18-2a7ef5c75ad0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:36.440000 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d63f06-9353-4ed5-bd18-2a7ef5c75ad0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:36.442113 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d63f06-9353-4ed5-bd18-2a7ef5c75ad0 service neutron] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:36.452223 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1436/2899] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:28:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 27 15:28:36.750360 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c6ccc2-a556-4c2c-91bc-7a98cb25ad6f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:36.751897 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-79c6ccc2-a556-4c2c-91bc-7a98cb25ad6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:28:36.765742 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c6ccc2-a556-4c2c-91bc-7a98cb25ad6f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:36.770498 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d974ea4-bb98-421b-ac07-effafde531de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c29995c0b564cd7879c58c75dc099cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fdd45226b7e4d17bd9b93b1a5b70196', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:36.771438 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d974ea4-bb98-421b-ac07-effafde531de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:36.771723 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d974ea4-bb98-421b-ac07-effafde531de admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:36.772002 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d974ea4-bb98-421b-ac07-effafde531de admin admin] PATH_INFO: `/v3/projects/8031cc1037dd48ebbf72c9bcc8ef46ba` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:36.773203 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d974ea4-bb98-421b-ac07-effafde531de admin admin] RBAC: Authorizing `identity:delete_project(project_id=8031cc1037dd48ebbf72c9bcc8ef46ba)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:36.775184 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d974ea4-bb98-421b-ac07-effafde531de admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:36.822665 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9d974ea4-bb98-421b-ac07-effafde531de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8031cc1037dd48ebbf72c9bcc8ef46ba', 'request_id': 'req-9d974ea4-bb98-421b-ac07-effafde531de'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:36.830023 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9d974ea4-bb98-421b-ac07-effafde531de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8031cc1037dd48ebbf72c9bcc8ef46ba', 'request_id': 'req-9d974ea4-bb98-421b-ac07-effafde531de'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:36.836051 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9d974ea4-bb98-421b-ac07-effafde531de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8031cc1037dd48ebbf72c9bcc8ef46ba', 'request_id': 'req-9d974ea4-bb98-421b-ac07-effafde531de'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:36.847163 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9d974ea4-bb98-421b-ac07-effafde531de admin admin] The token cache is being invalidate because project 8031cc1037dd48ebbf72c9bcc8ef46ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 27 15:28:36.848202 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9d974ea4-bb98-421b-ac07-effafde531de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:36.848663 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9d974ea4-bb98-421b-ac07-effafde531de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 27 15:28:36.850260 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1464/2900] 10.222.0.50 () {58 vars in 1232 bytes} [Fri May 27 15:28:36 2022] DELETE /identity/v3/projects/8031cc1037dd48ebbf72c9bcc8ef46ba => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 27 15:28:46.447733 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=30,keystone:DELETE=1 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:28:46.843906 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=50,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:28:53.368236 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d63f06-9353-4ed5-bd18-2a7ef5c75ad0 service neutron] Authenticating user token {{(pid=80394) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:28:53.369638 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-f7d63f06-9353-4ed5-bd18-2a7ef5c75ad0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:28:53.474039 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d63f06-9353-4ed5-bd18-2a7ef5c75ad0 service neutron] Validating token access rules against request {{(pid=80394) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:28:53.536164 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a92af4-c90d-48e4-ba7e-3f243007f8dd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8293dab3ffd647f5927d323572fdbdce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80394) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:28:53.537580 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a92af4-c90d-48e4-ba7e-3f243007f8dd service placement] REQUEST_METHOD: `GET` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:28:53.537863 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a92af4-c90d-48e4-ba7e-3f243007f8dd service placement] SCRIPT_NAME: `/identity` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:28:53.538137 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a92af4-c90d-48e4-ba7e-3f243007f8dd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80394) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:28:53.539931 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.fernet_utils [None req-47a92af4-c90d-48e4-ba7e-3f243007f8dd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80394) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:28:53.550952 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a92af4-c90d-48e4-ba7e-3f243007f8dd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:28:53.552606 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a92af4-c90d-48e4-ba7e-3f243007f8dd service placement] RBAC: Authorization granted {{(pid=80394) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:28:53.562990 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: [pid: 80394|app: 0|req: 1437/2901] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:28:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 27 15:29:03.558086 dv-cinder-842542-2-iscsi devstack@keystone.service[80394]: DEBUG dbcounter [-] [80394] Writing DB stats keystone:SELECT=67 {{(pid=80394) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 27 15:33:59.390647 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d974ea4-bb98-421b-ac07-effafde531de admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 27 15:33:59.392143 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9d974ea4-bb98-421b-ac07-effafde531de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 27 15:33:59.408256 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d974ea4-bb98-421b-ac07-effafde531de admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 27 15:33:59.414047 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e956cd5b-2ee3-498c-be50-35756a68ebce service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8293dab3ffd647f5927d323572fdbdce', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'daf12585428b41a2b45bdbc85413a6c4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 27 15:33:59.414047 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956cd5b-2ee3-498c-be50-35756a68ebce service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 27 15:33:59.414376 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956cd5b-2ee3-498c-be50-35756a68ebce service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 27 15:33:59.414569 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e956cd5b-2ee3-498c-be50-35756a68ebce service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 27 15:33:59.425370 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e956cd5b-2ee3-498c-be50-35756a68ebce service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 27 15:33:59.427043 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e956cd5b-2ee3-498c-be50-35756a68ebce service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 27 15:33:59.438082 dv-cinder-842542-2-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1465/2902] 10.222.0.50 () {60 vars in 1365 bytes} [Fri May 27 15:33:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0)